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An  Assessment  of  Interactive  Graf  cs  Processing 
in  Short-Range  Terminal  Wea'  Forecasting 


1.  INTRODUCTION 

The  practical  emergence  of  computer-driven  interactive  graphics  display  sys¬ 
tems  has  spurred  a  move,  in  both  civilian  and  military  terminal  weather  opera¬ 
tions,  from  a  manual  mode  to  a  more  automated  mode.  Conventional  weather  data, 
presently  available  in  weather  stations  via  teletype  and  facsimile,  can  be  accessed 
for  plotting,  analyzing,  manipulating,  and  displaying  in  virtually  an  unlimited 
number-  of  ways  by  resident  software  once  the  basic  data  are  ingested  into  the  com¬ 
puter  system.  Add  to  that  the  potential  and  routine  availability  of  imagery  from 
polar  orbiting  and  geostationary  weather  satellites,  conventional  and  Doppler  radar, 
and  other  remote  sensors  and  you  have  a  dramatic  increase  in  the  amount  and  rate 
at  which  information  can  be  made  available  to  the  forecaster-user. 

Because  of  the  amount  of  data  and  the  wide  range  of  options  available  to  the 
forecaster  to  manipulate  it  in  the  interactive  display  system,  careful  consideration 
must  be  given  to  the  methods  available  to  effectively  use  the  system.  As  was 
stated  in  the  first  report  on  this  study,  1  the  potential  exists  for  inundating  the  base 
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Range  Terminal  Forecasting:  An  Initial  Assessment,  AFGI.-TR -83 -0093 
AD  AH 7T 0  5.  - - - 


11 


t  -  t’hc  r  s'.,’  ior.  :'t>  r  ir.is!  e  r  a  ]  ’  h  more  i tor:: .. . !  i  •  > :  i  -a  I  1  ■. '  .  -u  s  ;  ge  ■ » j  i '  ion.—  '  1:  ' :  <  ■ .  i :  i 
In-  h.iivlli-  l.  i- s pec  i .ill y  ut.cri  In •  or  she-  is  trying  '<>  cope  a  r!i  i  i  ite  1  a:  I 

ever-changing  .icsIh-i-  situation.  Conversch  ,  if  ’he  system  :s  properly  sij-uc- 
>  u  :-t- J  ,ei  1  ’lie  lori-c  is’crs  have  heeti  ,  lequ .-. ’  e  ]\  •  !•  I  on  ;  >  s  us  nil-  u:i  ler  v  .rnus 
we  oiler  situations,  sue!:  svs'eins  offer  'he  oppo:  mui'  v  for  major  ,  Kniices  m 
v.  e  c  lie r  s'  n  n  ope  r.-.t  ions,  a  i  It-  -  r  i  nging  a  e  e  l;e  r  suppoi" ,  m  1  ’  e r m  i n  1 1  foreo  s' 
H’i'Ur  -.c\  . 

Idle  ISA!  h . :  s  emh.i  rkt-  1  on  -  base  '.H'.i'lier  staMoti  :::o  le  rn. .'  ion  p;  igr  »i;. 
e  Ale  !  Au'ona'e  i  We.i'her  Dis'  .'Abut  ion  System  (AW  US)  .'.hie!:,  .'..hen  fu!h  innAe- 
metire  .1,  a  ill  provi  le  base  level  suppoi"  '•>  oper. Perns  void  1a:  It-  oil  'o  leplo-.e  1 
Air  !-'o  :•<•(•  m  1  Army  !  ac’  :c.,  1  unir  s.  U'l'll  AW  US,  envi  ronrnent  A  1  ,<  .  .n  1  pro  111- " 
Ail!  In-  .inquire  1,  stored,  displace  l.  analyze  .1,  oil  forecast  through  n.o  lernr/e  1 
eonipute:-  ,.n  1  It-  lie  ,te  1  communication  s\ stems.  The  principal  a  c  nher  1.0.  i 
sourees  available  through  \W  US  will  he  provi  le  1  from  the  Air  Force  Global 
Wtn'hcr  C'en’ral  (Al-'GWC)  ,«n  1  the  Automate]  Weather  Network  (AW  \).  The  AWN 
roller's,  elits,  reformats,  and  transmits  AOather  lata  between  base  weather 
stations  worldwide.  Those  da'u  inelude  alphanumeric  product s  such  as  hourly  or. 
special  surface  weather  observations,  twice  -  l.iilx  raw  insonde  (upper -air)  observ. 
'ions,  plain  language  text  including  advisories,  terminal  forecasts,  oi  l  map  lis- 
eiissions,  which  ate  presently  received  at  base  weather  stations  via  the  I'OMhDS 
teletype  circuits.  Products  available  m  weal  her  stations  from  Al-'GWC'  include 
various  surface  and  upper-air  analysis  and  forecast  maps  that  are  received  via 
the  Air  Force's  facsimile  circuit  (AFI)IGS).  With  AW  US  these  data  products  will 
flow  directly  into  an  on-site  computer  facility  designed  to  store,  process,  and 
display  w  eather  data  fields  on  alphanumeric  ..nil  color  graphic  terminals.  Resi¬ 
dent  software  in  the  base  weather  station's  AW  US  computer  will  allow  the  weather 
man-user  to  request  the  execution  of  a  wide  :  inge  of  analysis  and  forecast  -guid¬ 
ance  procedures. 

With  AW  US  there  will  be  an  important  difference  in  the  formal  of  data  flowing 
from  AFGWC  in  that  objectively -determined  analysis  and  forecast  products  will 
be  Uniformly  G ridded  Data  Fields  (FGUF).  That  is,  the  individual,  regularly- 
sp.ir-H  gridpoin*  values  will  be  transmitted  to  base  weather  stations  where  they 
can  be  contoured  md  displayed  in  map  form,  as  required  by  the  weatherman-user 
In  addition,  lata  flowing  from  AFGWC'  will  include  vector  graphic  products  to 
describe  weather  maps,  charts,  and  figures  and  raster  scan  products  (initially 
limited  to  AFGWC's  Satellite  Global  Uatu  Base).  The  Satellite  Global  Data  Base 
(SGUB)  is  comprised  mainly  of  3-nm  resolution  visible  and  1R  satellite  imagery 
for  each  hemisphere.  Raw  imagery  input  to  the  SGUB  comes  from  available 
pol  l  r  -or  biting  satellites  (for  example.  DA1SP  and  NOAA),  which  is  integrated  into 
the  SGUB  m  real-time  after  imagerv  from  each  quarter-orbit  is  received  at 


AFGWC.  At  any  one  point  in  time,  therefore,  the  individual  pixel  values  on  the 
global  SGDB  may  come  from  observations  made  several  hours  apart.  This  char¬ 
acteristic  greatly  limits  the  value  of  the  SGDB  in  mesoscale  or  short-range  fore¬ 
casting  applications. 

The  Me  I  DAS  facility  at  AFGL  has  access  to  most  of  the  data  that  will  be  avail 
able  with  AWDS.  (The  major  exceptions  being  TGDF  and  SGDB  from  AFGWC.) 
They  can,  however,  be  reproduced  in  MclDAS  to  a  large  extent  by  analyzing 
observational  data  and  forecast  guidance  from  the  National  Weather  Service  (NWS) 
In  addition,  other  data  sources  (for  example,  GOFS  imagery)  and  capabilities 
available  within  our  MclDAS  are  germaine  to  short-range  terminal  forecasting 
applications.  See  Chisholm  et  al*  for  a  description  of  MclDAS  and  its  residence 
software  capabilities. 

A  research  and  development  study  was  undertaken,  using  the  MclDAS  as  an 
AWDS  prototype,  to  examine  benefits  and/or  problems  inherent  in  video  display 
systems  for  the  prepartition  and  monitoring  of  short-range  terminal  forecasts. 

A  2-vr  mesoscale  forecast  experiment  (MFC)  was  conducted  to  assess: 

(1)  The  value  of  certain  mesoscale  objective  plot,  analysis,  and  forecast 
procedures  in  the  preparation  of  short-range  terminal  forecasts, 

(2)  The  relative  difficulty  in  preparing  certain  forecasts  (elements)  using  an 
interactive  graphics  system, 

(3)  The  value  of  certain  remotely  sensed  data  in  short-range  terminal  fore¬ 
casting,  and 

(4)  The  performance  of  forecasters,  in  weather  episodes  with  substantial 
mesoscale  variability,  in  generating  both  numerical  (deterministic)  and  probabil¬ 
istic  terminal  forecasts  using  an  interactive  graphics  system.  The  forecast  prep¬ 
artition  and  met -watching  aspects  of  the  experiment  were  structured  to  simulate 
the  process  and  requirements  of  providing  the  base  weather  station  support  stated 
above. 

2.  MESOSCALE  FORECAST  EXPERIMENT  (MFE) 

The  procedure  that  was  established  to  assess  aspects  of  interactive  systems, 
data  sources,  and  weatherman-user  effectiveness  was  to  conduct  a  forecast  test 
experiment  addressing  a  particular  set  of  short-range  terminal  forecasting 
requirements  using  research  meteorologists.  Two  MFK  test  periods  were  estab¬ 
lished;  the  first  was  conducted  in  the  summer  of  1982,  the  second  in  the  summer 
of  1983.  This  report  combines  the  results  of  the  1982  and  1983  tests;  an  initial 
assessment  based  on  the  1982  test  has  been  published  previously.  ^  In  order-  to 
conduct  the  tests  most  efficiently,  data  fr  om  significant  weather  episodes  were 


archived  during  the  1981-1982  and  1982-1983  winter  and  early  spring  storm 
seasons  in  the  Northeast  U.S.  The  archived  data  sets  were  then  restored  in 
MclDAS  for  each  foreca  experiment.  Twenty  archived  episodes  were  use  1  in 
the  experiment,  ten  during  the  summer  of  1982  and  ten  during  the  summer  of 
1983.  The  1983  episodes  are  described  in  Appendix  A  of  this  report,  the  1982 
episodes  in  Chisholm  et  al.  ^ 

The  MFK  tests  were  conducted  such  that  the  time  available  for  forecast  prep¬ 
aration  was  controlled  to  real-time  limits.  The  forecasters  had  a  singular  task 
and  objective  during  the  tests;  prepare  terminal  forecasts  for  two  locations  using 
as  many  of  the  resources  available  to  them  through  MclDAS  for  the  purpose  of 
evaluating  new  and  standard  products  and  data  sources.  One  forecast  experiment 
was  conducted  each  week  (typically  in  two  4-hr  periods  per  forecaster  and  with 
three  forecasters  working  together  to  evaluate  the  weather  situation  but  indepen¬ 
dently  preparing  their  forecasts).  At  the  conclusion  of  each  case,  each  forecaster 
completed  evaluation  forms  in  which  the  products  used  and  the  forecast  aspects  of 
the  case  were  assessed.  These  then,  formed  the  basis  for  addressing  purpose.-; 
(1),  (2)  and  (3)  of  the  MFK  as  described  in  the  INTRODUCTION. 

2.1  Experimental  Procedures 

The  forecasters  were  required  to  forecast  (on  tin  hourly  cycle)  windspee  l  and 
direction,  total  cloud  amount,  and  ceiling  height  for  periods  of  1,  2,  4,  and  n  hr 
ahead.  The  period  n  was  the  closest  interval  (greater  than  4  hr)  between  forecast 
time  and  0(1,  12,  18,  or  00  GMT.  In  addition,  fi-hr  quantitative  precipitation  fore 
casts  (QPF)  were  prepared  each  hour  for  the  "next"  full  fi-hr  period  ending  at  0(1, 
12,  18.  or  00  GMT.  Forecasts  for  two  airfield  locations  were  required  for  each 
case;  the  specific  locations  were  predicated  on  the  availability  of  FOBS  bulletins 
containing  model  output  statistics  (MOS),  I.FM-II  guidance,  and  3-D  trajectory 
forecasts  that  were  made  available  to  the  forecasters  for  guidance  purposes. 

L.ogan  International  Airport,  Mass.  (BOS),  being  the  closest  candidate  location  to 
AFGI.,  was  a  forecast  location  in  each  of  the  cases  used  in  the  test.  The  second 
location  varied  among  Bradley  International  Airport,  Conn.  (BDK),  Kennedy 
International  Airport,  N.1.  (JFK),  and  Green  Airport,  R.I.  (PVD),  depending  on 
factors  related  to  the  episode  being  tested. 

For  windspeed  and  direction,  a  numerical  or  deterministic  forecast  was  pre¬ 
pared.  With  each  of  the  other  elements,  both  categorical  or  numerical  forecasts 
and  probability  forecasts  were  prepared.  For  total  cloud  amount,  the  category 
(clear,  scattered,  broken,  or  overcast)  and  the  probability  of  occurrence  of  each 
category  were  required;  for  ceiling  height,  a  specific  height  category  and  prob¬ 
abilities  for  the  categories  listed  in  Table  1;  and  for  QPF,  the  (i-hr  precipitation 


amount  (in  inches)  and  the  probabilities  for  the  categories  listed  in  Table  2.  The 
category  breakdown  for  total  cloud  amount,  ceiling  height,  and  fi-hr  OP!'  arc 
compatible  with  categories  of  MOS  and  related  guidance. 


Table  1.  Ceiling  Height  Categories 


Category 

rn 

2 

3 

4 

Height  (100  ft) 

0-1 

2-4 

5-9 

10-29 

Table  2.  Six-hr  QPF  Categories 


Category 

1 

QPF  (in. ) 

0-0. 24 

d 

i 

0.  50-0.  99 

>1.  00 

The  computer-based  forecast  entry  and  verification  procedure  referred  to  as 
the  Mesoseule  Forecast  Facility  (IUFF)  is  a  "menu-driven"  interactive  system 
designed  to  ingest  individual  forecasts  into  a  data  file  through  the  use  of  formatted 
interrogation !  response  messages  via  the  Ale  1  DAS'  .keyboa  rd  alphanumeric  CUT 
terminal  interface.  Figure  1  depicts  a  sample  message  in  its  interrogation  format 
(a)  and  response  format  (b).  Fach  of  the  principal  participants  in  the  experiment 
had  reserved  storage  in  AlclDAS  in  which  individual  forecasts  and  verification 
statistics  were  accumulated. 

The  assessment  forms  completed  after  each  case  are  shown  in  Appendix  It , 
The  purpose  of  the  Alesoscale  Forecast  Variable  Assessment  was  to  establish  the 
relative  difficulty  in  preparing  forecasts  of  each  element  and  to  try  to  isolate  the 
reason(s)  for  the  difficulties  encountered.  An  assessment  form  was  completed 
for  each  of  the  forecast  elements.  The  Product  Usefulness  Assessment  was  used 
to  measure  the  relative  value  in  terminal  forecasting  of  the  objective  display  fea¬ 
tures  in  Mel  DAS,  especially  the  new  ones  created  for  this  experiment.  The  fore¬ 
casters  were  directed  to  judge  the  most  and  least  useful  products  (for  the  case 
just  completed)  and  give  reasons  why  they  were  so  rate  1. 

After  each  case,  the  numerical,  categorical,  and  probability  forecasts  were 
verified  •••ithin  the  RIFF,  accumulated  statistics  were  summarized  and  made  avail¬ 
able  to  the  forecasters  for  review.  Figure  2  is  a  sample  of  the  accumulate  1 
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Figure  1.  Kxample  of  Menu-Driven"  Interactive 
Procedure  Used  for  Forecast  Fntry  into  Automated 
Verification  Phase  of  RIFF:  (a)  Interrogation  For¬ 
mat  and  (h)  Response  For  mat 


verification  statistics  compiled  for-  each  forecaster.  Separate  statistics  were 
computed  for  predictions  verified  for  Logan  (BOS),  which  are  denoted  PRIMARY, 
for  the  other'  station  (denoted  SECONDARY),  and  for  the  combined  verification. 
The  error  statistics  calculated  were:  mean  absolute  error  and  rmse  for  numer- 
cal  and  categorical  forecasts  and  the  p-score  and  cumulative  p-score  and  Heidke 
skill  score  vs  persistence  for  probability  forecasts. 

The  method  of  comparison  for  the  study  was  persistence,  measured  directly 
and  in  sample  (unconditional)  climatology  form.  The  Heidke  skill  score,  which 
measures  the  percent  difference  (improvement)  of  verification  scores  relative  to 
a  set  of  control  forecasts,  was  calculated  using  persistence  as  the  control.  The 
rms  vector  error  results  were  used  to  calculate  the  Heidke  skill  score  for  wind 
forecasts,  and  the  cumulative  p-score  was  used  for  the  other  forecast  elements. 
The  equations  for  calculating  the  verification  scores  are  described  in  Chisholm 
et  al.  The  persistence  probability  forecast  was  generated  by  (a)  directly  assign¬ 
ing  a  probability  of  1.0  to  the  category  that  pertained  at  forecast  (initial)  time  and 
a  probability  of  0.0  to  the  categories  that  did  not  pertain  and  (b)  using  sample 
climatology  statistics.  The  latter  was  not  determined  for  QPF  because  of  its 
limited  sample  size.  For  total  cloud  amount,  the  unconditional  2-yr  sample 
climatology  was  clear  (0.0a),  scattered  (0.08),  broken  (0.07),  and  overcast 
(0.80).  For  ceiling  height  it  was  category  1  (0.02),  2  (0.01),  8  (0.11),  4  (0.30), 

5  (0.20),  and  0  (0.30),  where  the  category  numbers  are  those  defined  in  Table  1. 

10 
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Figure  2.  Kxample  of  Verification  Statistics  Compile  i 


Hach  climatology  is  based  on  the  240  observations  that  comprised  the  conditions 
that  existed  at  forecast  (initial)  times  for  the  20-episode  Ml'F  sample. 

Although  not  used  for  direct  comparisons  in  the  experiment  (that  is,  compared 

by  means  of  skill  score),  t  he  Generali  zed  l'xponential  Markov  (GFM)  technique 

was  adapted  to  Mr  IDAS  and  evaluated  on  the  20-case  sample.  The  GFM  technique 

is  a  fundamental  statistical  w  eather  forecasting  procedure  recent  Iv  developed 

•> 

through  'he  pioneering  research  of  Miller.”  Miller  defines  GIM  .s  "a  statistic  il 
technique  for  predicting  t he-  probability  distribution  of  local  surface  weather  ele¬ 
ments  hour  by  hour.  It  uses  only  the  current  local  surface  weather  conditions  as 
predictors.  From  these  probability  distributions,  categorical  predictions  are 
made  for  each  surface  weather  element."  For  its  use  m  the  Mi  l  ,  the  GIM 
technique  was  adapter!  to  MclDAS  to  generate  wind,  (  loud  cover,  and  ceiling  height 
forecasts  for  both  forecast  locations,  which  were  veiified  at  1-,  2-,  4-,  and  ti-hr 
intervals  coincident  with  M  FF-forecas'er  verifications.  We  ad  ipted  the  mini  - 
computer  version  of  G  FM  to  MclDAS,  which  unfortunately  could  not  be  easily 
adapted  to  the  variable  nature  of  our  ti-hr  forecast.  For  'hat  reason,  only  the 
1-,  2-,  and  4 -hr  GFM  results  will  be  presented.  Figure  2  is  an  example  of  a  GFM 
forecast  for  P.OS  during  fine  forecast  episode-.  Three  things  must  he  recognized 
regarding  the  application  of  GFM  in  the  Midi,  f  irst,  GFM  is  founded  on  a  Markov 
assumption  (that  is,  the  future  state  is  completely  determined  by  the  present  state 
and  is  independent  of  the  way  in  winc  h  the  present  state  has  developed).  Second, 
it  uses  multivariate  linear  regression  equations  that  were  developed  from  contin¬ 
uous  observational  samples  that  spanned  a  10-yr  period  ( 1  0a4  - 1  Mi  a)  at  a  number 
of  locations  and,  as  such,  are  dim  ttologic  illy  and  statistically  sound.  Third,  it 
was  applied  to  cases  in  the-  Ml  F  that  represented  "heavy  weather1'  and  lo  not, 
therefore,  reflect  the  characteristics  of  the-  full  sample  from  which  the  GFM 
statistical  operators  were  developed.  I'  ,s  felt,  however,  that  GFM's  universal 
and  easy  applicability  m  dr-  i'  proper  nd  appropriate  to  include  its  performance 
in  the  qualitative  assessment  of  the  Mi  l.'. 

Forecast  guidance  information  derived  from  N'MC's  t.FM  (FOL'S  Bulletins) 
was  available  to  the  forecasters  in  teletype  form  luring  the  1082  forecast  exper¬ 
iments  and  was  available  directlv  from  Mr  IDAS  -luring  the  If '82  forc->  d  experi¬ 
ments.  It  included  the  MGS  forecasts  (FOFS  12),  ’  I.FM  Guidance  forecasts 
(FOUS  fiO  -78),  ^  and  2-D  trajectory  forecasts  ( FOt  S  aO-aT).  ’  The  \  M  C  guidance 
information  is  ingested  into  Mr  IDAS  via  the  FA  A  WB-804  da«a  link,  then  decode  1 
and  formatted  for  display  md  analysis.  The  guidance  information  is  available  to 
the  forecasters  in  a  listing-type  form  that  can  he  viewed  on  the  alphanumeric 
terminal  or  a  hardcopy.  Many  of  the  guidance  parameters  can  be  m.dy/.ed  and 

References  2  through  a  will  not  he  listed  here.  See  References,  page  80. 
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viewed  on  the  color'  monitor.  Kxamples  of  tlie  analyzed  guidance  products  arc 
shown  in  Figures  4  through  0.  The  forecasters  have  the  option  of  viewing  one 
analysis  on  the  entire  screen  (with  the  ability  of  overlaying  another)  or  a  fnur- 
panel  analysis  displaying  one  parameter  ovei  four  forecast  periods  or  four 
parameters,  which  is  useful  in  determining  guidance  suggesting  movement  and 
intensity  changes.  The  verification  of  MOS  was  limited  to  the  forecasts  which 
verified  at  Oil,  12,  18,  or  00  GMT. 
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Figure  4.  I.FM  Guidance  (FOX'S  CO -7H).  Surface 
pressure  analysis  (0  PS)  and  (l-hourlv  forecasts 
(fi  PS,  12  PS,  'and  18  PS)  from  1  March  1983. 

12  GMT:  units  in  mb  deviation  from  1000  mb 


2.2  Test  Sample 

The  weather  episodes  selected  for  study  in  t he  Mi  l!  were  1001-1082  and 
1982-1983  winter-early  spring  storm  events  occurring  over  the  Northeast  F.S. 

To  be  considered  as  a  forecast  experiment,  a  case  must  have  met  two  basic 
requirements.  First,  a  12-  to  24-hr  period  of  significant  weather  in  the  form  of 
changing  cloud  cover,  ceiling  height  or  winds,  and  or  the  occurrence  of  precipi¬ 
tation  in  southern  New  England  was  required  to  fully  test  the  utility  of  an  inter¬ 
active  graphics  system  and  forecast  aids  in  preparing  short -ranee  mesosc.de 
forecasts.  Second,  a  complete  24-hr  data  set,  consist  mg  of  convent  ion.il  hourly 
surface  observations,  upper-air  data,  satellite  imagery,  manually  heitize!  r..  lar 
(MDR),  and  operational  computer  guidance  was  required.  This  woul  1  provide  to 
12  hr  of  data  for  pre-forecasting  familiarization  with  the  weather  situadon,  ti  hr 
for  forecasting  and  up  to  12  hr  for  verification. 

The  20  cases  chosen  for  the  MFF  were  selected  from  among  3  1  c,.ses  that 
had  been  archived.  They  comprised  three  general  synoptic  si*ust ions;  midwest 
cyclones,  New  Fngland  cold  fronts,  and  east  coast  evdogenes is.  e.udi  bringing 
significant  weather  changes  to  New  Fngland.  Table  3  lists  'he  Luo,  force  >st 
times,  forecast  stations,  and  a  brief  synoptic  descrip'ion  for  each  forecast  exper¬ 
iment.  It  should  be  noted  that  Boston  was  the  primary  force. .st  si.it  ion  for  dl 
20  experiments.  The  secondary  station  (PYD,  Bill.,  or  JFK!  was  chosen  e-cot  i- 
ing  to  the  completeness  of  that  station's  data  sets  an  I  !h<  po'cn'i  d  of  die  •  e  die: 
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Figure  5.  MOS  Guidance  (FOFS  12):  (a)  Proba¬ 
bility  of  Precipitation  (POP)  Forecasts  it  ii-hourl\ 
Intervals  (12,  18,  24,  and  30  hr)  From  1  March 
1083,  12  GMT,  Fnits  in  10-percent  Increments; 

(b)  C  eiling  [feight  (C1G)  Forecasts  at  ti-hourlv 
Intervals  ( ' >,  12,  18,  and  24  hr)  From  1  March 
1983,  12  GMT.  Fnits  in  100-ft  Increments 


Table  3.  MFK  Test  Cases  (Rrief  Description) 


Forecast 

experiment 

No. 

For 

Date 

1 

3  2 1 >  82 

2 

4/20  82 

3 

3  31  82 

4 

4  27  82 

■' 

3  11  82 

t; 

12  i:>  81 

7 

12  1  81 

8 

3  4  82 

(i 

12  '22  81 

10 

4 '(K82 

11 

10  25  82 

12 

11  '5  82 

13 

1113  82 

14 

1  15  83 

1 5 

2  3  83 

in 

2  11  83 

17 

1  15/83 

18 

2  23  83 

19 

3  ,/2  '83 

•ast 

Times 

Forecast 

Stations 

14-19  7. 

POS  JFK 

19-007 

POS  PVD 

05-107 

POS  PVD 

14-197 

POS  JKK 

18-23  7 

POS  PVD 

18-23  7 

POS JKK 

17-22  7 

POS JFK 

17-22  7 

POS  PVD 

00-117 

POS  PDF 

04-097 

POS  PDF 

15-20  7 

BOS  PVD 

15-207 

BOS  'PVD 

N 

1 

o 

BOS  JFK 

12-177 

BOS  JFK 

14-197 

BOS  BDK 

10-217 

BOS/PDL 

23-047 

BOS /BDK 

10-217 

BOS/  PVD 

13-197 

BOS/ PVD 

Forecast  Situation 


Cold  front  moving 
through  New  Kngland 

Weak  frontal  wave  mov¬ 
ing  toward  New  Kngland 

Approaching  pre-front al 
<■  loud  band 

Cold  front  moving 
through  New  Kngland 

Appr  oaching  pre-frontal 
cloud  band 

Rapidly  deepening 
cyclone  approaching 
New  Kngland 

Warm  frontal  wave  ap¬ 
proaching  New  Kngland 

Cold  front  approaching 
New  Kngland 

Overrunning  precipi¬ 
tation 

Kxplosive  coastal 
cyclogenesis 

Occluded  cyclone  south 
of  New  Kngland 

Cold  front  moving 
through  New  Kngland 

Approaching  cold  front 
and  frontal  wave 

Major  coastal  snowstorm 

Midwest  cyclone  and  ap¬ 
proaching  cold  front 

Coastal  cyclogenesis 

Midwest  cyclone 

Cold  front  approaching 
New  Kngland 

Large  ocean  storm, 
significant  rainfall 

Weak  frontal  wave  ap¬ 
proaching  New  Kngland 


20 


3  21  83 


15-20Z 


POS  DDL 
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Figure  7.  Twenty-four  Hourly  Observations  for  I.ogan  International  Airport  < I i <  >S ) 
for  the  Ten  1082  Cases  of  the  Mesoscnle  Fori cast  Experiment;  Hours  (!  to  11 

Hepresent  Forecast  Times.  qoF®^*  (tl1  001,1  ml  height  (hun-lrels  of  feet). 


ceiling  height  (hutrlre  ls  of  feet). 


TNI.  utilimite-1.  ()]?S  obscure;!).  QPI  'i-ltr  prn  i  p  1 1 . 1 1  i  r  >  n  amoim*  in  niche; 
inti  \VN  Weather  svmbols  ('.vinHflags  in  knots) 


FORECAST  EXPERIMENT  (BOS) 
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»  1st  FORECAST  MADE  AT  6th  HOUR  »*  LAST  FORECAST  MAOE  AT  llth  HOUR 

Figure  8.  Twenty-four  Hourly  Observations  for  I.ogan  International  Airport  (BOS) 
for  the  Ten  1P83  Cases  of  the  Mesosoale  Forecast  experiment;  Hours  n  to  11 

Represent  Forecast  Times.  CC'H  ceiling  height  (hundreds  of  feet. 

I'NL.  -  unlimited,  OBS  -  obscured),  QPF  fi-hr  precipitation  amount  in  inches, 
and  WX  r  Weather  symbols  (windflags  in  knots) 
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Figure  9.  Twenty-four  Hourly  Observations  for  Secondary  Station  (BDL,  JFK,  or 
PVD)  for  the  Ten  1982  Cases  of  the  lUesosoale  Forecast  experiment;  Hours  (i  to  11 


Represent  Forecast  Times.  qbf®vvx  ^CH  --  ceiling  height  (hundreds  of  feet. 


FNl.  unlimited,  OBS  obscured),  OPF  ti-hr  precipitation  amount  in  inches, 
and  \VX  Weather  symbols  (windflags  in  knots) 


H(/>  >omxo  ~n 


3.  MFE  1982  83  TEST  RESULTS 


The  major  aspects  of  'he  MFE  dealt  with  (1)  an  evaluation  of  the  usefulness 
of  certain  display  products,  (2)  an  assessment  of  the  relative  difficulty  in  fore¬ 
casting  certain  weather  elements  and  the  products  and/or  data  sources  that  were 
found  to  be  useful  to  those  forecasts,  and  (3)  a  statistical  evaluation  of  forecaster 
performance.  The  results  of  these  aspects  are  presented  in  Sections  3.  1,  3.2, 
and  3.  3,  respectively.  There  may  appear  to  be  a  certain  amount  of  over  lap  in  the 
discussions  presented  in  Sections  3.  1  and  3.2.  By  necessity,  however,  the  results 
of  the  forecast  difficulty  assessment  are  predicated  on  the  availability  and  utility 
of  pa rticulur  products  in  regard  to  the  problem  of  forecasting  specific  weather 
elements. 

3.1  Product  Assessment 

In  the  course  of  the  1982-1983  MFE  tests,  the  participating  forecasters  were 
encouraged  to  use,  to  their-  fullest  advantage,  any  and  all  of  'he  data  sources, 
display  products,  and  analysis  forecast  techniques  available  within  Me  I  DAS  in  the 
preparation  of  their  forecasts.  In  that  regard  they  were  specifically  instructed  to 
use  each  of  the  new  techniques  at  least  once  during  each  test  episode.  In  so  doing, 
it  was  felt  that  a  r  easonable  and  fair  assessment  of  these  aspects  of  Mo!  DAS  w  ould 
be  accomplished. 

Table  4  lists  certain  aspects  of  the  assessment  process  for  products  that 
were  used  frequently.  The  "product  key-in"  refers  to  the  keyboard  instruction 
that  activates  the  generation  of  a  discrete  product  on  Me  IDAS.  Those  products 
listed  in  Table  4  without  an  asterisk  are  part  of  the  previously  existing  Me  IDAS 
suite  of  available  routines.  Their  primary  function  is  listed  below: 

IA  -  Geographic  (map)  plot  of  weather-  vanable(s)  on  the  color  monitor; 
used  with  either  surface  or  upper-air  data;  can  include  one  to  four- 
panels  per  screen;  can  he  plotted  over  satellite  imagery 

ZJ  -  Decoded  listing  of  surface  observations  on  alphanumeric  terminal 

(for  example,  by  stations  in  a  state  for  a  specific  time  or  a  time  ser  ies 
for  a  specific  station) 

ZK  -  Surface  analysis  using  one-pass  Cressman  technique  on  color  monitor; 
ideal  for  broad -view  quick-look  assessment 

MR  -  Contoured  geographic  analysis  of  manually-digitize  1  ra  1st-  (Ml)l!)  on 

color  monitor;  displayed  independently  or  in  conjunction  with  concurrent 
satellite  imagery  md  or  surface  obse rv  a  ions 

ZP  -  Map  plot  of  one  v.ti  .ble  on  the  alphanumeric  terminal 


ENT/ET  -  GOES  IR  imagery  color  enhancement  based  on  defined  tempera¬ 
ture  thresholds  (in  the  case  of  ET,  the  user  defines  the  threshold) 
KZ  -  Surface  analysis  using  five-pass  Barnes  technique  on  color  monitor; 
retains  more  mesoscale  detail 

PF  -  2-D  trajectory  forecast  model  based  on  analyzed  rawinsonde  observa¬ 
tions 


FI  -  Decoded  display  of  FOES  bulletins  (MOS,  l,FW  Guidance,  and  3-D 
trajectories)  on  alphanumeric  terminal 
BS  -  Listing  of  the  GEM  technique's  forecast  displayed  on  the  alphanumeric 
terminal  or  the  printer  (hardcopy) 

FK  GUI  -  Analysis  of  several  LEM  guidance  forecast  (I-’Ol'S  GO-78)  param¬ 
eters  on  color  monitor  (parameters:  700-mb  vertical  velocity, 
relative  humidity,  lifted  index,  1000-500  mb  thickness,  boundary 
layer  streamlines,  boundary  layer  windspeed,  boundary  layer 
temperature,  sea-level  pressure,  fi-hr  precipitation  amount) 

FK  MOS  -  Analysis  of  several  MOS  forecast  (FOES  12)  parameters  on  color 
monitor  (parameters:  precipitation  probability,  thunderstorm 
probability,  snow  amount  category,  temperature,  dewpoint  tem¬ 
perature,  surface  streamlines,  windspeed,  cloud  category,  ceiling 
category,  visibility  category) 

FK  TRA  -  Analysis  of  several  3-D  trajectory  forecasts  (FOES  30-57)  on 

color  monitor  (parameters:  temperature  and  dewpoint  temperature 
at  700  mb,  850  mb  or  surface;  K-index) 

FA  -  Plot  of  trajectory  forecast  on  color  monitor  indicating  origin,  G-hourly 
position  and  rising  or  sinking  motion  along  the  path  for  a  parcel 
terminating  at  a  station  at  the  surface,  850  or  700  mb 
MS  -  Station  model  time  series  display  on  color  monitor;  depicts  surface 
observations  for  up  to  six  stations  for  up  to  G  hr 
EC  -  Algorithms  to  estimate  cloud  cover  and  1-hr  precipitation  amount  from 
GOES  visible  and  IR  imagery  displayed  on  alphanumeric  terminal 
KY  -  Epper-air  analysis  (constant  pressure  surface)  using  a  two-pass  Barnes 
technique  on  color  monitor 

ST  -  Individual  variables  line  or  bar  graph  time  series  display  on  color 
monitor 

ZC  -  Contours  a  previously  analyzed  and  stored  grid  array  and  displays  it 
on  the  color  monitor 

rP  -  Log  p-skew  T  display  of  rawinsonde  observations  on  color  monitor 
rS  -  Surface  analysis  using  five-pass  Barnes  technique  on  color  monitor. 

Uses  a  time-space  weight  function  in  which  1-  and  2 -hr  old  observations 
are  advected  downstream  and  included  in  the  analysis 


Ft'S  -  Menu-driven  forecast  decision-assistance  procedures  using  the  alpha¬ 
numeric  terminal 

The  second  column  indicates  the  approximate  ''wall-clock"  time  (in  seconds) 
it  takes  Mel  DAS  to  generate  a  product  from  the  time  the  "key-in"  action  is  re¬ 
quested  to  when  the  product  is  fully  displayed.  Since  lUolDAS1  graphics  levice 
is  considerably  slower  than  state-of-the-art  devices  and  its  CPF  is  a  24-bit 
machine  without  hardware  multiply -divide  capability,  one  can  presume  substanti¬ 
ally  shorter  generation  time  can  be  achieved.  In  <  ach  case  these  times  reflect 
the  creation  of  products  on  the  local  scale  typically  used  in  this  experiment  (that 
is.  as  generated  by  Me  IDAS  state  and  city  displays).  The  generation  of  national 
or  regional  (for  example,  eastern  US)  maps  generally  increases  the  times  indi¬ 
cated  hy  a  factor  of  .'1  or  4.  Obviously,  none  of  these  routines  are  very  time  con¬ 
suming  in  that  most  are  executed  in  well  under  liO  sec. 

The-  new  routines,  tailored  to  increase  the  depiction  of  mesoscale  information 
do  not  result  in  substantial  increases  in  wall-dock  time.  The  KZ  analysis  takes 
HO  sec  longer  than  the  ZK  analysis.  This  increase  results  from  going  to  a  five- 
pass  procedure  that  recovers  a  Large  portion  of  the  mesoscale  detail  suppressed 
in  the  one-pass  technique.  Neither  of  the  tailored  plot  routines  (MS  and  GT)  are 
time  consuming  (35  and  20  sec,  respectively).  Fvcn  the  multi-step  procedure  of 
remapping,  analyzing,  and  displaying  M  OH  data  (MR  key-in)  takes  less  than  a 
minute  while  the  application  of  the  forecast  guidance  technique  based  on  2-D 
trajectories  (PF)  and  Satellite  algorithms  (FOR)  has  a  combined  wall-clock  time 
of  25  see.  Note  that  key-ins  that  use  the  black  and  white  alphanumeric  terminal 
(Zl,  PF.  ZP.  and  FI)  require  5  sec  or  less  wall-clock  time. 

The  "Number  of  (  sos"  column  reveals  the  extent  to  which  each  key-in  was 
invoke  1  by  the  forecasters  Juring  the  20-episode  tost  period.  The  overwhelming 
popularity  of  the  key-in  IA,  indicates  its  all-around  use  both  early  in  the  forecast 
process  when  large  amounts  of  data  are  digested  to  gain  an  overall  understanding 
of  the  evolving  synoptic  situation,  and  later  in  the  forecast  process  when  the  fore¬ 
casters  concent  rated  on  the  mesoscale  aspects  that  would  result  in  hour-to-hour 
weather  changes.  The  IA  kev-in  was  used  early  in  the  forecast  process  in  con¬ 
junction  with  the  Zl  and  ZK  key-ins  (second  and  third  most  used,  respectively). 
Two  typical  four-panel  displays  used  by  the  forecasters  during  this  mesofr'  noptic 
learning  process  are  shown  in  Figures  11  and  12. 

f  igure  11  is  a  four-panel  display  of  the  weather  situation  that  existed  at 
1200  GMT,  15  January  1983.  It  shows  (a)  surface  pr  essure  analysis  with  over- 
laved  wind  flags,  (b)  ceiling  height  plot,  (c)  cloud  amount  plot,  and  (d)  an  M Dll 
analysis  with  over  laved  weather  symbols.  Due  to  the  poor  resolution  of  the  ha rd- 
ropv  capabilities,  this  figure  is  shown  here  as  four  separate  figures  although  it 
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Figure  11.  Four-panel  Analysis  of  General  Weather 
Situation  (Regional):  (a)  Pressure  (mb  Deviation 
from  1000  mb)  and  Surface  Windflags,  (b)  Ceiling 
Heights  (Hundreds  of  Feet),  (c)  Havered  Cloud  Amount 
Categories  (From  the  Left;  Low,  Middle,  and  High 
Cloud  Amount;  0-Clear,  1-Scattered,  2-Broken,  and 
3-Overcast),  and  (d)  MDR  Analysis  With  Plotted 
Weather  Svmbols 
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Figure  12.  Four-panel  Surf:ice  Piossui  o  Analysis  at  2 -hr 
Intervals  (mb  Deviation  From  1000  mb) 


would  be  viewed  by  the  forecasters  as  a  four-panel  display  on  one  screen.  Fig¬ 
ure  12  is  a  four-panel  display  of  the  surface  pressure  analysis  every  2  hr,  begin¬ 
ning  at  1200  GMT.  These  plots  and  analyses  would  he  displayed  on  regional  map 
backgrounds  (state).  The  first  four-panel  display  (Figure  11),  accomplishes  the 
task  of  familiarization  with  the  current  situation,  and  the  second  (Figure  12,  whir 
could  just  as  well  have  been  a  display  of  M  DR  or  ceiling  height  plots  every  2  hr) 
informs  the  forecaster  of  the  movement  and  intensity  changes  in  the  system  over 
the  past  several  hours.  The  7.1  key-in  was  used  to  view  several  hours  of  observe 
tions  at  a  particular  station  or  to  view  data  in  a  particular  region  at  one  observa¬ 
tion  time. 

In  the  later  stages  of  the  forecast  process,  the  forecasters  tended  to  use  the 
I A  and  K7,  (seventh  most -used)  key-ins  on  a  mesoscale  map  background  (city )  to 
zoom  in  on  the  mesoscale  aspects  of  the  forecast  situation.  Here-  again,  four- 
panel  lisplavs  of  several  forecast  par  ameters  (a  combination  of  TVs  an  1  K/.'s)  or 
one  parameter  over  the  past  4  to  8  hr  was  used  to  view  all  forecast  parameters  ■ 
once  or  to  follow  the  movement  or  intensity  changes  of  a  particular  parameter 
over-  the  past  several  hours. 


ZP,  the  fifth  most-used  key-in.  was  called  upon  most  often  when  huu  -p.tn  l 
analysis  was  displayed  on  the  color  video  monitor  and  a  forecaster  h  -!•  <■  1  , 
''quick-look"  <5 -sec  generation  time  on  the  alphanumeric  video  screen)  vcrv 
specific  point  of  interest  without  having  to  clutter  erase  the  font  -panel  lispl  i\. 
Typical  ZP  displays  were  3-hr  pressure  changes,  ti-hr  prec ipit  c ion  amount, 
numerical  plot  of  windspeed  and  direction,  and  ceiling  an  1  we.Pher  svmbol  pin*. 

In  the  middle  and  later  stages  of  the  forecast  process,  tnc  forecasters  use  1 
the  tailored  products  (Ml?,  KZ.  MS.  GT)  and  various  guidance  pro  lucts  0  1,  I  K 
Gl'I,  PF,  CC.  etc.),  to  focus  on  specific  aspects  of  the  present  force. i.- 1  pi  .hlc::. 
and  to  consult  guidance  for  the  longer-  range  force. is-  s. 

LFM  products  (FI.  FK  Gl  1,  FK  M()S.  l'K  TliA,  an  1  FA)  and  (1KM  .u-i  e  m  •  i<- 
available  as  forecaster  guidance  on  Me  IDAS  for  the  1083  expernnen' s,  lore,  a'r 
found  the  listings  of  the  I.FM  guidance  forecast  (FOCS  m  l  MtiS  (l't)T  S  id) 

output  to  be  valuable  guidance  for  the  longer  forecast  intervals  (4  m  1  n  hr). 
Analyses  of  LFM  output  were  used  minimally  although  the  m  dvsis  of  it  w  c  .-o"H  - 
times  good  ;is  guidance  for  storm  development,  movement,  an  I  position  for  the 
later  forecast  periods.  The  relatively  large  number  of  uses  for  the  GI'M  gui  1  •i.-' 
(BS)  must  be  tempered  by  the  fact  'hat  the  forecast!  ;  conclu  le  1  that  r  gave  lr’b 
useful  information  in  rapidly  changing  situations  of  the  type  used  in  these  tcs*.-. 

The  limited  use  of  other  tailored  product s  (TP,  TS,  Ft  S)  was  ill  ribitte  i  to 
several  different  factors.  The  routine  to  display  log  p-ske.v  T  soundings  of 
radiosondes  (TP)  was  found  to  be  of  limited  value  in  short-range  mrsoscalc  fore¬ 
casting  because  of  a  lack  of  timeliness  (soundings  taken  only  every  12  hr)  an.l  poor 
horizontal  representativeness.  The  time-space  surface  analysis  technique  (TS). 
which  uses  a  forecaste r-chosen  motion  vector  to  advect  data  from  the  previous 
2  hr  to  be  included  in  the  analysis,  was  often  found  to  produce  inconsistent 
analyses.  Forecasters  used  it  mostly  for  ceiling  analyses  which,  due  to  its  lis- 
continuous  nature  spatially,  yielded  analyses  that  the  forecasters  could  have  lit  *  le 
confidence  in.  The  cold  front  decision  assistance  procedure,  PCS,  was  generally 
judged  to  be  too  time  consuming  for  the  forecasters  who  wore  all  quite  famili.it- 
with  MclDAS's  interactive  capabilities.  Recall,  however,  that  the  long-range 
intent  of  developing  such  procedures  was  for  use  in  training  forecasters  new  to 
particular  region  or  to  using  an  interactive  system. 

The  use  of  GOT'S  imagery  (visible  and  II?)  was  extensive  throughout  the  fore¬ 
cast  experiments,  especially  for  storms  developing  and  moving  up  the  east  coast 
where  a  large  portion  of  the  developing  and  advect  ing  w  eather  w  ..s  out  over  the 
water,  where  conventional  data  coverage  was  sparse  a*  best.  The  conventional 
GOF.S  imagery  display  options,  which  include  animation  (time  series  looping), 
channel  switching  (alternating  visible  and  IK  images  to  evaluate  cloud  layering), 
color  enhancement  of  brightness  and  11!  gray  scales  an  I  overt  w  ing  convent  ton  d 
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analyses  .md  data  plot?,  were  used  extensively.  This  capability  is  judged  to  be 
central  to  effective  short -range  terminal  forecasting.  The  only  aspect  of  satellite 
imagery  the  forecasters  were  specifically  asked  to  assess  was  a  procedure  for 
enhancing  the  IK  images  (F\'T  FT)  according  to  temperature.  The  MXT  <  KT  key- 
ins  (sixth  most  used)  were  found  to  be  useful  to  study  the  movement  and  develop¬ 
ment  of  weather  systems,  particularly  in  tracking  the  evolution  of  substantial 
precipitation  areas  within  the  storm  system  as  identified  by  cloud  top  tempera¬ 
tures  below  a  specific  value(s). 

The  information  presented  in  Table  4  under-  the  heading  "  Usefulness1'  sum¬ 
marized  the  concensus  assessment  of  the  forecasters  on  the  relative  usefulness 
in  terminal  forecasting  of  the  products  and  data  sou  rets  available  to  thorn.  The 
numbers  shown  reflect  the  total  number  of  times  a  participating  forecaster  judged 
a  product  most  (or  least)  useful.  (See  first  page  of  Product  Usefulness  Assess¬ 
ment  in  Appendix  B  for  form  completed.  )  There  were  up  to  six  forecasters  par¬ 
ticipating  in  each  of  the  20  test  episodes,  thus  there  was  the  potential  of  120 
responses  in  each  category  (most  and  least).  In  fact,  a  total  of  110  assessments 
were  completed  because  several  episodes  involved  less  than  six  forecasters. 

The  IA  data  plot  was  found  to  be  the  most  useful  product  evaluated  (Table  4). 
Most  often  used  as  a  four-panel  display,  the  1A  key-in  was  used  extensively  in 
both  the  early  learning  stages  of  the  forecast  process  and  the  later  stages  when 
the  forecasters  were  most  interested  in  timing  anil  accuracy.  Forecasters  found 
the  l A  key-in  so  useful  because  a  large  amount  of  data  could  be  presented  quickly 
in  a  four-panel  display  showing  four  different  variables  at  one  observation  time 
or  one  variable  every  1  or  2  hr  to  track  the  movement  or  intensity  changes  of 
various  fields.  The  I A  key-in  was  also  used  regularly  in  overlaying  surface 
observations  upon  satellite  imagery. 

The  IMS  (station  model  time  series  plot.  Figure  13)  was  found  to  be  the  second 
most  useful  product.  The  use  fulness  or  value  of  the  MS  routine  was  arrived  at 
despite  the  fact  that  many  other  products  were  used  in  the  experiments  to  a  much 
greater  extent  (Table  5).  This  illustrates  the  potential  value  of  having  routines 
available  that  allow  the  forecaster  to  tailor  displays  to  the  specifics  of  the  weather 
situation  he  or  she  is  dealing  with.  The  display  exhibited  several  qualities  found 
useful  by  the  forecasters.  The  fi-st  is  the  ability  (and  ease)  to  specify  the  stations 
to  be  included  in  the  cross-section  and  to  tailor  it  to  the  episode  under  considera¬ 
tion.  The  second  is  that  it  provides  a  wealth  of  basic  information  central  to  the 
terminal  forecast  problem  in  a  format  that  permitted  extensive  subjective  inter¬ 
pretation  to  track  one  or  more  elements  and  define  its  spatial  extent.  Forecasters 
found  if  most  useful  for  the  1-  to  4-hr  forecast  interval  of  the  type  required  in 
base  weather  stations  in  support  of  aircraft  arrivals  and  local  area  requirements, 
and  in  tracking  wind  shift  lines  (fronts)  and  the  leading  or  trailing  edge  of 
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The  products  found  to  be  least  useful  are,  beginning  with  the  least  useful, 
GKM  (BS  key-in),  MDR  (MR  key-in),  log  p-skew  T  (TP),  and  the  time-space 
analysis  technique  (TS).  Forecasters  found  GKM  did  not  depart  sufficiently  from 
persistence,  even  in  active  weather  regimes.  Not  surprisingly,  it  is  not  adept  at 
forecasting  the  onset  or  the  end  of  precipitation.  A  typical  example  of  G Kill's 
inability  to  depart  from  persistence  sufficiently  to  simulate  the  observed  weather 
associated  with  an  approaching  or  departing  storm  is  shown  in  Figure  10.  Fig¬ 
ure  10(a)  shows  that  as  worsening  weather  approaches.  Gill's  ceiling  forecast 
follows  persistence  in  the  1  and  2  hr  forecasts,  then  lowers  the  ceiling  somewhat 
(to  3000  ft),  not  nearly  enough  to  account  for  the  very  low  ceilings  (hazardous  to 
aviation)  that  were  observed  in  the  0-hr  forecast  period  (400  ft).  With  improving 
conditions  Figure  10(b)  GFM  forecast  persistently  low  ceilings  when  the  ceiling 
was  observed  to  improve  dramatically.  GKM  actually  decreased  the  coiling  in 
the  4-  and  ft— hr  forecast  periods  when  no  ceiling  was  observed.  As  can  be  seen 
from  Figure  1 1  > ,  GKM  forecasts  were  quite  poor  in  predicting  the  onset  and  end 
of  precipitation.  In  general,  GKM  did  not  forecast  precipitation  until  it  had  be¬ 
gun.  The  value  of  its  wind  guidance  is  diminished  by  broadly  defined  wind  direc¬ 
tion  and  speed  categories. 

The  negative  assessment  of  MDR's  usefulness  (in  the  product  assessment.)  is 
the  result  of  its  inconsistent  availability.  During  several  episodes,  as  precipi¬ 
tation  approached  the  forecast  stations,  key  MDR  stations  li  1  not  report  until  the 
precipitation  began.  This  resulted  in  erratic  and  unrealistic  analyses  that  ser¬ 
iously  diminished  the  ability  to  forecast  the  movement  and  intensity  oT  appro  ach¬ 
ing  precipitation  areas.  In  the  episodes  with  reliable  and  continuous  MDR  lat  i, 
the  MR  kev-in  was  used  frequently  and  was  found  to  provide  goo  1  qualitative 
information  for  the  QPK  and  ceiling  height  forecasts.  On  the  other  hand,  the 
results  shown  in  Table  5  (discussed  in  the  next  section)  indicate-  that  MDR  1  a  > 

3!> 


was  useful  in  the  QPF  forecast  70  percent  of  the  time.  This  suggests  that  althoug 
MDR  data  availability  is  often  a  problem,  valuable  information  on  precipitation 
coverage,  movement,  or  intensity  can  be  attained  from  1UDR  analyses  during  an 
episode. 


WORSENING  CONDITIONS  !  IS  FEBRUARY  1983,  BOS1 
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Figure  10,  Example  of  GEM  Ceiling  Height  and  Weather  Forecasts  vs  Obser¬ 
vations  for  1 5 _F ebrua ry  1983  for  BOS 


3.2  Forecast  Difficulty  Assessment 

The  second  aspect  of  the  forecaster  assessment  of  the  Ml-'l!  dealt  with  tlie 
terminal  forecasts  themselves.  The  forecasts  that  were  prepared  included  wind- 
speed  and  direction,  total  cloud  cover,  ceiling  height,  and  fi-hr  QPF  for  intervals 
of  1,  2,  4,  and  n  hr  (10  hr  or  less  based  on  criteria  cited  in  Section  2.  1).  These 
elements  and  intervals  were  selected  to  represent  the  primary  forecast  responsi¬ 
bility  in  base  weather  station  support  to  aircraft  takeoff  landings  and  local  area 
missions.  After  each  case,  each  forecaster  completed  a  Mesoscale  Forecast 
Variable  Assessment  form  (shown  in  Appendix  B),  for  each  of  the  four  variables. 
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Table  .*>  pre  sents  'Ik*  l  it  ,  tabulate  1  f coir,  the  Forecast  Variable  Assessment 
forms  suggf-sung  quali'ativi  1\  whirl:  proinrts  ivnilnblo  to  the-  forecasters  on 
MrlDAS  were  mos’  (  m  l  le  ts’)  useful.  I'  shows  some  interesting  results  concern¬ 


ing  the  usefultK-. 


lit.,  sources  for  each  forecast  element  const  lore  1. 


Details  on  tlteir  usefulness  will  be  present eil  later  in  this  section,  but  some  of  the 
gene  r. dines  a  re: 

(1)  Satellite  la’a  is  shown  to  be  most  useful  for  the  QPF  and  cloud  amount 

forecasts,  while  provi  ding  useful  information  more  often  than  not  for  the  ceiling 
heigh'  forecast .  11  .-.as  occasionally  useful  for  wind  forecasts  through  the  timing 

of  frontal  w  mdshifts  .m  l  'he  location  an  1  movement  of  coastal  developments  and 
their  associ  pe  l  windspeed  and  lireetion  changes, 

(2)  Manually  Hgin/o  1  radar-  is  shown  to  be  useful  about  70  percent  of  the 
time  tVe-  OP!  for  e  ■  . st .n  I  2a  percent  of  the  time  for  ceiling  height  forecasts. 
MrlDAS  plo»  and  .n -.lysis  routines  (IA,  7.K,  KZ)  were  found  to  be  useful  more 
often  than  not  for  all  the  forecast  parameters  while  being  useful  nearly  always  for 
•  he-  wind  and  ceiling  height  forecasts. 

(3)  Guidance  for  ecasts  (I.FM  guidance,  MOS,  trajectory,  and  GEM  forecasts) 
were  useful  for  all  forecast  parameters  more  often  than  not  -while  being  most 
useful  for  the  QPF  and  wind  forecast,  and 

(4)  The  2-D  trajectory  model  (PF)  showed  limited  use  for  all  forecast  param¬ 
eters  mainly  because  it  depends  on  raw  insonde  winds,  which  lose  value  as  one 
departs  from  00  12  GMT. 

Table  f>  presents  .statistics  on  the  forecast  difficulty  assessment.  Sixty-one 
of  the  111  respondents  judged  ceiling  height  forecasting  most  difficult,  27  judged 
ti-hr  QPF  most  difficult,  14  judged  windspeed  and  direction  most  difficult  while 
cloud  amount  was  judged  to  be  most  difficult  by  nine  respondents.  Conversely, 

44  of  111  respondents  judged  total  cloud  amount  the  easiest  of  the  four  parameters 
to  forecast,  30  judged  fi-hr  QPF  easiest,  17  judged  windspeed  and  direction,  and 
no  one  judged  ceiling  height  easiest  of  the  four  variables  to  forecast.  Thus,  for 
this  set  of  20  winter-early  spring  weather  episodes,  the  forecasters  determined 
the  ranking  of  the  forecast  parameters  to  bo.  fr  om  most  difficult  to  easiest: 

(1)  ceiling  height,  (2)  fi-hr  QPF,  (3)  windspeed  and  direction,  and  (4)  total 
cloud  amount  (Table  7). 

In  each  case  the  forecasters  were  also  asked  to  indicate  the  forecast  interval 
beyond  which  they  had  little  confidence  in  their  forecasts.  For  ceiling  height  that 
interval  averaged  2.  4  hr  (for  the  20  experiments)  ranging  from  0  to  (’>  hr.  For 
<i-hr  OP!  it  averaged  5.2  h-  and  ranged  from  0  to  IB  hr.  For  windspeed  and 
direction  it  averaged  4.3  hr  and  ranged  from  1  to  12  hr.  For  total  cloud  amount 
it  aver  aged  ti.  4  hr  and  varied  from  1  to  If!  hr.  These  results  confirm  the  ranking 
of  ceiling  height  as  the  most  difficult  parameter  to  forecast  and  total  cloud  amount 


I  .Me  7.  Forecast  P  i  i-.um-'c!'  Difficulty  \ssi 


F=- 


Forec ..s' 

P  ir.,mer  er 


Ceiling 


Wincl^ 

1  7  -hr  Q  PF 

Cloud  Amount 


i  Most 

I 

j  I  Ji  ri'icuP 

t  r  r 

i- ;r~hr 

id  i  28 

27  !  3  4 

i  r>  |  in 


Kt.sicsi 


.3  1  1 

-I - j  _ - 

1 7  |  0 

'  72  17 

.  20  '  30 

i 

i  22  '  74 


Hours  of 
Con:-:  deuce 


2.  4 


7.4 


.is  *he  easiest.  The  somewhat  contradictory  confidence  mtetvai  for  i'-hrQP! 
(considering  its  ranking  as  second  most  difficult)  is  a  result  of  the  great  varia¬ 
bility  in  'he  episodes  regarding  the  difficulty  in  preparing  the  precipitation  fore¬ 
cast.  This  is  also  evident  in  the  distribution  shown  in  Table  7,  where  the-  fore¬ 
caster- replies  for  the  ti-hr  (J PF  forecast  are  well  distributed  from  most  difficult 
to  least  difficult. 

The  difficulty  in  accurately  forecasting  cloud  cc-iling  heights  at  airfields  has 
been  recognized  for  a  number  of  years.  The  inherent  variability  of  storm  system 
low  clouds  (in  space  and  time),  their  sensitivity  to  local  factors  (terrain,  water 
bodies,  etc.),  'he  limits  in  our  ability  to  observe  and  report  on  their  character¬ 
istics,  and  the  limits  in  our  understanding  of  the  complex  physics  by  which  they 
evolve,  all  serve  to  diminish  the  extent  to  which  cloud  ceiling  can  be  forecast. 
Short-range  terminal  forecasting  is  especially  dependent  on  the  observational 
component  and  it  was  deficiencies  in  that  component  that  created  'he  most  difficult; 
•'or  'he  forecasters.  While  half-hourly  GOES  satellite  imagery  substantially  in¬ 
creases  obser  vational  data  on  storm  system  evolution,  it  provides  only  inferential 
jam  ori  ceiling  conditions  at  best.  Several  of  the  episodes  occurred  a'  ight  when 
only  IK  imagery  was  available  thereby  limiting  the  extent  to  which  forecasters 
could  try  to  deduce  lower  cloud  conditions  through  an  integrated  evaluation  of 
visible  ,.nd  infrared  imagery.  On  more  than  one  occasion  low  stratus  ceilings 
adverted  in'o  the  forecast  locations  from  the  ocean  areas  to  the  cast,  and  south 
under  a  middle  and  or  high  cloud  shield  that  precluded  the  detection  of  the  stratus 
in  the  GOES  imagery.  Two  other  nighttime  observation  shortfalls  complicated 
'he  ..lreadv  difficult  forecast  problem.  First,  certain  surf  ice  observation  loca¬ 
tions  operate  on  ..  limited  daily  schedule  in  which  oper  uions  are  curtailed  a1 
nigh'  t  for  example,  mi  Jnigh*  to  07  LST).  Second,  some  oc.-c  rvers  fill  to  detect 


and/or  report  changes  in  cloud  conditions  in  a  timely  manner  (that  is.  'he 
''sunrise  special"  observation  was  not  uncommon)  thereby  a  iling  mot  >  ’  muse" 
to  the  naturally  variable  time  and  spare  char  n-ter  of  low  clou  Is  in  these  kinds  o.' 
weather  systems. 

Table  5  indicates  that  MclIJAS  plot  and  analysis  routines,  especially  the  pi  v 
of  ceiling  height  and  weather  through  the  I A  key -in.  and  the  bar  graph  lisp!  ay  of 
cloud  observations  through  the  GT  key-m.  were  most  useful  overall  for  ceiling 
forecasts.  The  four-panel  time  series  iisplay  of  ceiling  height  (IA  four-pane! 
plot)  was  relied  on  heavily  in  determining  ceiling  height  trends  an  1  also  m  en¬ 
forcing  the  extent  to  which  clou  1  fields  were  irregularly  iis' ribute  1,  w  l:n  i,  -a! 
in  flatter  probability  forecasts.  Satellite  imagery,  useful  bout  •'.!>  per. o‘  t he 
time  for  ceiling  height  forecasts  (Table  <i).  provided  valuable  inform  dor  .■  e 
movement  of  cloud  areas  and  the  leading  ,n  1  trailing  edges  of  'dou  l  slue;  1  .... 
Satellite  imagery  w..s  particularly  useful  in  predicting  lowering  ceilings  ,u.-t  i 
by  low  stratus  adverting  inland  from  over  the  ocean.  ]  OFS  AIOS  gui  1  mee  n:  .- 
vided  useful  information  on  the  longer  term  ceiling  conditions  more  often  'a  t  • 
The  trajectory  model  (PI  key-m)  occasionally  provide-'!  useful  information  • 'i.r 
trend  of  ceiling  heights.  Shortcomings  in  the  trajectory  model  were  attributed  'n 
evolving  wind  fields  diminishing  the  representativeness  of  the  trajectories  m  i 
terrain-related  cloud  patterns  that  were  n o'  translated  effectively.  Information 
on  the  ceiling  could  sometimes  be  inferred  from  Ml)l?  data.  In  summary  then, 
the  forecasters  used  the  IA  mesosoale  [)lo>  (city  map  background),  GT  bat  graph, 
and  satellite  imagery  for  the  I-  to  4-fn-  forec  asts  and  (lie  FOl'S  M OS  guidance. 

2 -D  trajectory  model,  IA  regional  plot  (state  map  background),  and  satellite 
imagery  time  loops  for  the  longer  ••  ange  forecasts. 

GOTiS  satellite  imagery,  MI)I1  analyses,  and  the  satellite  t  rajectorv-h  ased 
forecast  guidance  ( PF  CO,  products  no'  expecte  d  to  be  immediately  available 
within  the  initial  A1VDS  configuration,  were  found  to  bo  especially  useful  in  ti-hr 
QPF  forecasting  (Table  a).  The  integrated  displays  of  present  weather  and  11011 
analyses  superimposed  on  the  GOFF  imagery  provided  i  visual  confirmation  of  the 
portions  of  the  cloud  systems  'hat  are  producing  preoipit.it ion.  Then  satellite 
imagery,  with,  or  with.ou'  MOP  analyses,  can  be  use  I  effectively  to  track  the  pro¬ 
gression  of  rail'  areas  in  or  ler  to  time  its  arrival  at  forecast  locations.  W  hen 
the  storm  system  w  is  out  over  the  Atlantic,  satellite  imagery  provided  the  onlv 
valuable  source  of  current  inform  .lion.  In  the  .bsonce  of  hourly  r  onfall  amounts 
the  intensity  changes  in  'he  visible  .nd  II?  imagery  were  used  to  approximate  the 
ti-hr  QPF  totals.  The  color  enh  inrement  of  II?  imagery  base  1  >n  temperature 
thresholds  (K\T  FT)  ude  1  in  evaluating  the  growth  or  decay  of  rain-pro  lueing 
cloud  masses.  Sequencing  (looping)  or  the  color  enhanced  h  df-liourlv  im.igerv 
for  a  2  to  hr  period  was  ,  commonly  use  1  nrorc  litre  for  resolving  the  onset  and 


duration  estimates  at  the  forecast  locations.  During  daylight  portions  of  cases, 
the  forecasters  were  able  to  use  both  visible  and  1R  loops.  Nighttime  periods 
were  obviously  limited  to  IR  only  but  for  precipitation  forecasting  purposes,  it 
provides  valuable  guidance  by  itself. 

The  ability  to  remap  the  MDIl  analysis  and  geographically  overlay  it  on 
concurrent  GOKS  imagery  and  to  then  plot  observations  (for  example,  present 
weather  symbols)  on  both,  demonstrated  some  of  the  real  potential  inherent  in 
computer-driven  interactive  color  display  systems.  Figure  17  is  an  example  of 
such  an  integrated  display  for  1  March  1983.  In  a  forecast  situation,  these  fig¬ 
ures  would  be  viewed  in  an  animated  time-series  loop.  This  particular  loop  shows 
the  steady  northward  movement  of  precipitation  and  its  increasing  intensity  indi¬ 
cated  bv  the  expanding  DV1P  level  2  area.  Figure  17  shows  a  four-panel  Ml)!! 
analysis  with  overlayed  weather  symbols  (2 -hr  interval).  The  integrated  real¬ 
time  description  (nowcast)  these  analyses  present  to  an  aviation  or  local  area 
forecaster,  especially  when  sequenced  through  several  hours  in  an  animated  time- 
series  loop,  can  provide  the  subjective  basis  for  improved  understanding  of  the 
evolving  weather  system  and  for  improved  short-range  forecasts.  The  fore¬ 
caster's  ability  to  understand  the  complexity  of  the  system  (and  to  more  properly 
forecast  it)  can  be  seriously  limited  by  the  lack  of  MDIl  and,  in  particular, 
GOFS-type  satellite  imagery. 

The  satellite  -t  rajertory  precipitation  guidance  technique  (C'C)  was  the  fourth 
most  useful  product  to  the  forecasters  (Table  4).  It  was  useful  as  a  first  guess 
in  both  timing  the  arrival  or  departure  of  precipitation  and  determining  prccipit  i- 
t i on  amount.  The  trajectory-algorithm  guidance  is  only  applicable  under  daytime 
conditions  (needs  both  visible  and  111  imagery)  and  must  be  constrained  to  the 
central  75  to  80  percent  of  tin-  daytime  regime  due  to  the  sensitivity  of  the  algo¬ 
rithms  to  solar  angle  variations. 

Without  s  itelli'e  and  or  radar  data,  a  forecaster  must  rely  on  hourly  precipi¬ 
tation  intensity  observations  (light,  moderate,  heavy,  etc.)  as  the  basis  for 
departing  from  I  (U  S  or  other  rent  rally-generated  guidance.  There-  are  times, 
unfortunately,  when  'lie  hourly  intensity  values  can  be  quite  inconsistent  with 
3-  or  U-hourly  amounts.  The  station  model  time-series  display  (MS)  was  found 
to  he  most  useful  in  tracking  the  leading  or  hack  edge  of  the  precipitation  shied  1, 
particularly  when  used  m  conjunct  ion  with  four-panel  displays  (1A)  to  define  the- 
geographic  precipitation  distribution.  The  1.1- Ml -based  precipitation  guidance- 
(FOIS  'lO-f.FM  Guidance  ,nd  FOI  S  12-MOS)  pr  ovided  excellent  guidance  on  the 
precipitation  potential  of  the  storm  system;  guidance  which  the  forecasters  could 
adjust  to  information  on  actual  storm  tracking  and  intensity.  The-  fi-hr  QPF  FOI  S 
guidance  was  found  to  be-  more  useful  than  the-  guidance  for  the  other  forecast 
variables  (  Table  5). 
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Figure  17.  Example  of  Integrated  Disr’ay  of  GOES  Imagery,  MDH 
Analysis,  and  Plotted  Weathe r  Symb  ,  or  1  March  1  r> 8 3 :  (a)  1730, 
(b)  i930,  (c)  2130,  and  (d)  2330  GMT.  MDR  and  weather  symbols 
remapped  to  satellite  geography 
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Like  ceiling  height,  terminal  area  winds  can  be  variable,  particularly  during 
an  evolving  winter  storm  or  cold  front  situation,  and  they  can  be  especially  Sen¬ 
sitive  to  local  factors  (terrain,  water  bodies,  etc.).  While  the  1-min  mean  wind 
observation,  taken  each  hour  for  aviation  purposes,  better  serves  the  needs  of 
flying  safety,  it  has  been  shown  through  an  analysis  of  wind  spectra^  that  a 
30-G0  min  mean  wind  vector  and  some  measure  of  variability  would  be  more 
appropriate  for  short-range  prediction  purposes.  Unfortunately,  the  hourly 
1-min  mean  winds  contain  considerable  high-frequency  variance,  which  repre¬ 
sents  noise  in  the  prediction  process.  While  supplemental  cloud  and  precipitation 
related  observations  can  be  obtained  from  satellite  imagery  and  MDK  analyses  for 
data  void  regions  (especially  over  water),  remote  sensing  sources  are  not  pres¬ 
ently  available  for  surface  or  near-surface  winds. 

The  mesoscale  plot  and  analysis  routines  (IA,  MS,  KZ,  etc.)  were  found  to  be 
the  most  useful  products  in  resolving  the  location  and  movement  of  mesoscale/ 
synoptic  scale  wind  perturbations  such  as  inland  convergence  zones,  sea  breeze 
fronts,  frontal  boundaries,  mesohighs  and  lows,  isallobaric  centers,  etc.  ,  which 
aided  in  resolving  the  terminal  wind  forecasts  (Table  5).  Typically,  forecasters 
would  generate  a  detailed  surface  pressure  and/or  streamline  analysis  with  over- 
laved  wind  flags  (KZ  and  IA)  to  gain  an  initial  understanding  of  the  wind  distribu¬ 
tion  and  to  locate  mesoscale  features  such  as  mesohighs  or  surface  troughs. 

They  would  then  use  a  four-panel  surface  pressure  or  streamline  analysis  with 
overlayed  wind  flags  (2-hr  intervals,  every  2  hr)  to  determine  movement  and 
intensity  changes  of  the  mesoscale  and  synoptic  scale  features.  The  station 
model  time  series  (MS)  was  also  used  by  constructing  an  alignment  of  stations 
from  the  forecast  station  through  a  particular  feature  depicted  in  the  surface 
analysis  to  a  station  perhaps  50  to  100  km  beyond.  The  combination  of  these 
products  provided  the  necessary  time-space  representation  of  wind-related  fea¬ 
tures  that  could  be  obtained  given  the  limits  of  available  observations.  The  sur¬ 
face  and  boundary  layer  windspeed  and  direction  forecasts  presented  in  the  FOUS 
bulletins  (FOUS  12-MOS  and  FOUS  GO's)  were  found  to  provide  useful  guidance 
(Table  5),  especially  for  the  4-  and  n-hr  forecast  intervals.  GOES  satellite 
imagery,  MDR,  and  the  2-D  trajectory  forecast  model  were  of  little  value  to  the 
wind  forecasting  aspects  of  this  experiment  (Table  5). 

Clearly,  total  cloud  amount  (or  cloud  cover)  was  found  to  be  the  easiest 
variable  to  forecast  during  these  winter-early  spring  east-coast  storm  situations. 
The  relative  ease  herein  was  aided  by  the  availability  of  half-hourly  updates  of 
visible  and  IR  imagery.  In  most  situations  it  provided  all  the  information  needed 

G.  Muench,  H.S.  (1982)  An  Appraisal  of  the  Short-Range  Forecast  Problem 
Using  Power  Spectra,  AFGL-TR-82-0353,  ADA129315. 
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to  prepuce  the  cloud  cover  forecast.  When  the  middle  and  high  cloud  leading  edge 
of  a  storm  system's  cloud  shield  advanced  into  the  forecast  region  during  night¬ 
time  periods,  inconsistencies  between  satellite  IR  depictions  of  cloud  cover  and 
concurrent  surface  observations  of  sky  conditions  would  occasionally  exist. 
Invariably  this  was  attributable  to  untimely  surface  observations  that  nonetheless 
provided  some  uncertainty  in  the  forecast  process  in  which  some  forecasters 
tried  to  factor  " observer -deficiencies"  into  their  cloud  category  probability  fore¬ 
casts.  Timing  and  duration  of  cloud  cover  characteristics  were  generally  re¬ 
solved  through  repeated  time  series  looping  of  visible  and/or  IR  digital  imagery. 

The  most  widely  used  non -satellite  products  for  tracking  total  cloud  amount 
conditions  were  a  four-panel  plot  of  total  cloud  amount  (every  2-hr,  I  A),  the 
station  model  time  series  (MS),  and  the  bar  graph  display  of  cloud  observations 
(GT),  particularly  for  trucking  the  leading  (trailing)  edge  of  overcast  and  ceiling 
conditions.  The  forecasts  for  the  4  and  n  hr  periods  were  often  finalized  after 
giving  consideration  to  the  FOUS  (MOS)  guidance. 

3.3  Forecast  Verification  Statistics 

Numerical  and  probability  forecasts  were  generated  for  two  locations  for  20 
east -coast  storm  episodes  over  two  winter  seasons  (1081-1932  and  1982-1983). 
Much  episode  involved  up  to  six  forecasters,  each  preparing  terminal  forecasts 
each  hour  (for  six  hr)  of  cloud  cover,  ceiling  height  and  surface  wind  vector  for 
intervals  ranging  from  1  to  10  hr  and  G-hr  QPF.  A  computerized  verification 
procedure  was  implemented  to  accumulate  and  update  forecast  verification  statis¬ 
tics  on  each  forecaster  shortly  after  each  case  was  completed.  Updated  statis¬ 
tics  were  provided  to  participating  forecasters,  in  he  form  illustrated  in 
Figure  2,  prior  to  the  next  forecast  experiment  in  order  to  provide  timely  feed¬ 
back  on  individual  and  group  performance.  A  total  of  109  forecaster-days 
occurred  during  the  two  MFKs,  which  translates  into  a  total  of  1308  terminal 
forecasts  being  generated  of  each  variable  for  each  forecast  interval  (109  Y  0 
forecasts/case  X  2  stations/case). 

Objective  terminal  forecast  guidance  was  available  to  the  forecasters  from 
three  sources:  MOS,  LFM  Guidance,  and  GKM.  In  the  1983  experiment  they  were 
accessible  through  McIDAS  via  simple  keyboard  entry  procedures  while  in  1982 
they  were  available  only  in  hardcopy  form.  MOS  and  I. I'M  Guidance  forecasts 
could  be  displayed  on  the  alphanumeric  screen  in  teletype  message  form  or  could 
be  displayed  in  analyzed  map  form  on  the  color  monitor  in  either  single  panel  or 
four-panel  format  based  on  forecaster  preference.  The  four-panel  MOS  format 
is  illustrated  in  Figure  5. 
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It  was  noted  earlier  that  the  nature  of  MOS  limits  evaluation  of  it  to  the  six- 
hourly  intervals  its  forecasts  are  tailored  to  (00,  00,  12,  and  18  GMT).  As  such, 
the  verification  statistics  compiled  in  these  tests  for  MOS  were  limited  to  its  par¬ 
ticular  verification  times  and  were  stratified  into  1-,  2-,  4-,  and  n-hr  forecasts 
consistent  with  the  structure  of  the  MFK.  For  example,  a  MOS  forecast  valid  at 
1800  GMT  (for  example,  a  (>-hr  forecast  f-om  the  1200  GMT  model  run)  would  he 
verified  as  a  4-hr  forecast  for  the  MFK  forecasts  generated  from  1400  GMT 
observations,  a  2 -hr  forecast  for  MFK  forecasts  generated  from  1000  GMT  obser¬ 
vations,  and  an  1-hr  forecast  from  1700  GMT.  In  other  words,  we  used  it  con¬ 
sistent  with  its  operational  availability  in  order  to  quantify  its  guidance  value  in 
evolving  terminal  forecasting  applications. 

3.3.1  XFMKRICAL  (DKTKRMINISTIC)  FOHKCASTS 

Figures  18  to  20  summarize  the  rnise  statistics  of  the  numerical  or  determin¬ 
istic  predictions  of  surface  wind  (Figure  18),  cloud  cover  (Figure  19),  and  ceiling 
height  (Figure  20),  respectively.  Part  (a)  shows  results  for  each  experiment 
(MFK  82  and  MFK  83)  and  the  combined  set  of  forecaster  consensus  (MFK  82-83), 
where  the  numbers  in  parenthesis  are  the  percent  improvement  relative  to  a  per¬ 
sistence  error  for  each  sample.  Part  (b)  compares  overall  forecaster  perform¬ 
ance,  persistence,  and  GKM  for  the  20-episode  data  set  (MFK  82-83).  In  the 
case  of  ceiling  height  (Figure  20),  the  MFK  82  results  have  not  been  included  due 
to  a  change  in  the  numerical  forecast  parameter  from  the  specific  height  value  (in 
1982)  to  a  categorical  forecast  in  1983.  The  evaluation  of  the  1982  version  w  as 
rejected  because  the  procedure  formulated  to  treat  "no  ceiling"  forecasts  and 
observations  introduced  artificially-large  height  errors  values.  Therefore, 

Figure  20(a)  has  been  excluded  and  it  contains  just  the  MFK  83  (not  MFK  82-83) 
results. 

In  each  figure  the  results  are  accumulated  over  all  episodes  for  both  forecast 
locations,  for  all  forecasters  as  a  group,  for  persistence  and  for  GKM.  In  Fig¬ 
ures  18(b),  19(b)  and  20  the  range  in  individual  forecaster  performance  is  indicated 
bv  the  vertical  bar  through  each  group  mean  verification  statistic.  It  represents 
the  range  from  worst  to  best  individual  performance  over  the  full  extent  of  the  ex¬ 
periments).  Recall  that  the  n-hr  forecasts  ranged  in  length  from  5  to  10  hr:  the 
length  being  determined  by  the  interval  from  "forecast"  time  to  the  verification 
time  of  00,  Oli,  12,  or  18  GM  T.  No  attempt  has  been  made  to  further  separate 
the  n-hr  forecasts. 

Consistent  with  the  MFK  82  results,  the  forecasters  outperformed  persistence 
by  an  impressive  margin  for  each  forecast  variable,  especially  at  the  4-  and  n-hr 
intervals.  Although  forecaster  error  increased  with  forecast  length  (as  would  be 
expected  in  rapidly  changing  winter  storm  situations),  persistence  error  grew 
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Figure  18.  Hoot  Mean  Square  Krror  (HMSF)  Results  for  Meso 
scale  Forecast  Experiment  (MFF)  Wind  Forecasts:  (a)  All 
Forecasters  (Consensus)  and  (b)  Compared  lo  Persistence  and 
Generalized  Exponential  Markov'  (GEM)  for  MEE  1P82-1P83 
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Figure  20.  Root  Mean  Square  Krror  (RMSF)  Results  for  Meso- 
scale  Forecast  Hxperiment  (MFF)  Ceiling  Height  Forecasts: 

1983  Results  of  All  Forecasters  (Consensus),  Persistence,  and 
GFM 

even  faster  resulting  in  widening  percent  improvement  statistics  for  the  longer 
intervals.  The  comparison  of  1982  results  to  1983  for  wind  forecasts 
[Figure  18(a)|  would  suggest  that  the  1983  episodes  selected  for  testing  comprised 
windier  conditions  (higher  rmse  scores).  Relative  to  persistence  though,  fore¬ 
caster  skill  remained  about  the  same.  With  cloud  amount,  however,  persistence 
error  from  1  -  to  n-hr  was  nearly  identical  in  1983  as  compared  to  1982.  The 
better-  percent  improvement  scor  es  for  the  1-  and  2-hr  forecasts  [Figure  19(a)] 
can  be  attributed  to  the  implementation  of  an  improved  cloud-cover  tracking 
procedure  that  improved  the  short-range  timing  forecast  on  cloud  category  extrap¬ 
olation.  This  procedure  can  be  implemented  with  a  light  pen  option  on  an  inter¬ 
active  graphics  system.  In  its  absence  (as  was  the  case  with  McIDAS),  a  return 
to  basics  using  hardcopy  maps  and  the  manual  placement  of  successive  (hourly) 
boundaries  can  be  employed.  The  larger  forecaster  error  at  n  hr  in  1983  is 
attributed  to  cases  in  which  clearing  was  forecast  prematurely  too  often.  There 
was  a  fairly  substantial  range  in  forecast  skill  among  the  participating  forecasters, 
for  each  element  and  forecast  length.  This,  despite  the  fact  that  all  forecasters 
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had  access  to  the  same  resources.  In  this  regard,  consideration  of  the  relation¬ 
ship  between  forecaster  experience  and  skill  is  addressed  in  Section  3.3.4. 

The  GEM  did  not  generate  better  forecasts  than  persistence  for  wind  and  cloud 
cover  but  did  for  ceiling  height  during  these  winter  storm  episodes.  Except  for 
the  1-  and  2-hr  wind  forecasts,  the  difference  between  them  is  not  significant. 

Bear  in  mind  though  that  initial  conditions  at  forecast  time  were  typically  much 
more  degraded  than  the  climatological  norm  on  which  GEM's  statistical  operators 
were  developed.  Further,  the  MFE  cases  generally  involved  prolonged  periods 
of  inclement  weather.  Due  to  its  Markov  basis,  GEM  would  tend  to  forecast 
conditions  back  towards  normal  thereby  tending  to  increase  its  error  relative  to 
persistence. 

The  fi-hr  precipitation  amount  forecast  error  for  the  20  MFE  82-83  episodes 
(for  the  forecasters  as  a  group)  was  0.21  in.,  ranging  from  0.  17  to  0.20  in. 

Given  the  weather  situations  chosen  to  test,  the  modest  range  in  individual  fore¬ 
cast  skill  among  forecaster's  probably  reflects  the  greater  emphasis  on  the  use  of 
GOES  imagery,  MDR  and  FOES  guidance,  each  of  which  provided  more  quantitative 
information  on  precipitation  amount  than  they  did  for  the  other  forecast  variables. 

3.3.2  PROBABILITY  FORECASTS 

Probability  forecasts  were  generated  for  total  cloud  amount  categories  (clear, 
scattered,  broken,  and  overcast),  for  ceiling  height  categories  (listed  in  Table  1) 
and  for  fi-hr  QPF  categor  ies  (Table  2).  In  each  case,  these  categories  correspond 
exactly  to  those  used  in  MO S  and  related  .NWS  probability  guidance.  The  results 
are  summarized  in  two  forms:  (1)  against  the  probability  of  persistence  being 
maintained  and  (2)  against  the  MEE  sample  climatology  (except  fi-hr  QPF). 
Persistence  probability  forecasts  resulted  in  assigning  a  value  of  1.0  (100  percent) 
to  the  category  existing  at  initial  (forecast)  time  and  a  value  of  0.  0  to  the  other 
categories. 

Figures  21  and  22  summarize  the  percent  improvement  in  cumulative  p-score 
(cp)  statistics  of  the  forecasters  vs  persistence  probability  for  cloud  cover  and 
ceiling  height,  respectively.  The  results  for  MFE  82  and  MFE  83  are  shown 
separately  and  as  a  combined  outcome.  The  results  of  GEM  applied  to  the  twenty 
case  sample  is  shown  for  comparison.  Here  again  we  find  substantially  better- 
results  vs  pe rs i stance  at  all  forecast  intervals.  While  there  was  little  difference 
in  the  ceiling  height  results  in  1083  vs  1982,  the  cloud  cover  results  w'-re  better 
in  1982,  especially  at  the  n-hr  interval.  The  recurring  problem  of  forecasters 
moving  cloud  systems  out  of  the  forecast  region  too  quickly  is  reflected  in  the 
uncharacteristic  dip  in  skill  (from  44  percent  at  4-hr  to  28  percent  at  n  lit  )  in  the 
1983  exercise.  In  its  probability  form.  GEM  yielded  20-30  percent  improvement 
vs  per  sistence  over  the  2-vr  experiment,  for  both  cloud  cover  and  ceiling  height. 
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Figure  21.  Percent  Improvement  in  Cumulative  p-score 
of  IUesoscale  Forecast  experiment  (MFK)  Cloud  Amount 
Forecasts  vs  Persistence  Forecasts  for  All  Forecasters 
(Consensus)  and  for  GKW 
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Figure  22.  Percent  Impr  ovement  in  Cumulative  p-score 
of  Mesoscale  Forecast  experiment  CUFF)  Ceiling  Height 
Forecasts  vs  Persistence  Forecasts  for  All  Forecasters 
(Consensus)  and  for  GP'Al 


The  results  of  the  fi-hr  QPF  probability  forecasts  were  48  percent  improvement 
in  1982,  58  percent  in  1983,  and  52  percent  over  MFK  82-83.  Over  iwo  seasons, 
individual  forecaster  skill  ranged  from  38  to  81  percent  based  on  cumulative 
p-score. 

Figures  23  and  24  summarize  forecaster  consensus  and  GFM  vs  the  two- 
season  sample  climatology.  The  cumulative  p-score  of  the  climatology  model 
does  not  vary  substantially  as  the  forecast  interval  increases.  Since  forecaster 
and  GEM  p -scores  do  degrade  with  increasing  forecast  interval,  the  percent 
improvement  of  forecaster  consensus  and  GFM  generally  increases  with  time  .,s 
seen  in  Figures  23  and  24.  Here  again  the  over -aggressive  forecasts  of  clearing 
skies  at  n-hr,  characteristic  of  some  of  the  1983  cases,  is  reflected  in  the 
" worse -than-climatology"  results  for  n-hr  cloud  amount. 
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Figure  23.  Percent  Improvement  in  Cumulative  p-score 
of  Mesoscule  Forecast  Exper  iment  (M  I  K)  Clou  1  Amount 
Forecasts  vs  Sample  Climatology  Forecasts  for  All  Fore¬ 
casters  (Consensus)  and  for  GFM 
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Figure  24.  Percent  Improvement  in  Cumulative  p-scorc 
of  Rlesoscale  Forecast  Experiment  (RIFF)  Ceiling  Heie;:: 
forecasts  vs  Sample  Climatology  Forecasts  for  all  Fore¬ 
casters  (Consensus)  and  for  GFR1 


3.3  RIOS  HFSFI.TS 

A  quantitative  evaluation  of  the  RIOS  forecasts  for  this  experiment  iv.ust  be 
viewed  with  .  good  deal  of  caution  because  of  the  limited  sample  sizes  available 
for  analysis  for  reasons  cited  earlier.  One  can  infer  little  more  than  ar. 
indication  of  comparative  value.  Figures  25  through  27  depict  rinse  statist  it  s 
for  MOS  and  persistence  for  wind,  cloud  cover,  and  ceiling  height,  respectively. 
The  persistence  errors  are  for  the  full  2-yr  test  sample  while  the  MOS  results 
are  for  those  instances  (the  number  of  forecasts  are  shown  in  pnremiiest  s)  wrier 
a  MOS  forecast  could  properly  be  made.  It  is  only  at  n-hr  that  these  numbers 
are  coincident.  .Note  that  ceiling  heigiit  results  are  for  1983  only.  Not  surpris¬ 
ingly  RIOS  yields  worse  results  than  persistence  at  1  and  2  hr.  Beat  in  mind 
though  that  1-  and  2 -hr  RIFF  forecasts  correspond  to  MOS  forecasts  based  on 
ini'i  tl  conditions  for  RIOS  that  existed  11  and  10  hr  before  verification  time, 
resoert i vol\ .  RIFF  for  ecasts  of  3 -hr  duration  and  longei  are  supported  tiy  RIOS 
guidance  forecasts  (of  wind,  cloud  cover,  and  ceiling  heigh'.,  at.  least)  that  im¬ 
prove  upon  persistence.  The  results  for  wind  forecasts  are  particularly  revealing 
in  regards  to  the  "stability"  of  the  RIOS  rm.se  with  increasing  time,  a  character  - 
is*ic  of  RIOS  noted  by  Rluench.  '  In  fact,  if  on<  compares  the  MOS  wind  results 
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Figure  25.  Root  Mean  Square  Error  (RMSE)  for  Model 
Output  Statistics  (MOS)  and  Persistence  for  Wind  Fore¬ 
casts 
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Figure  2fi.  Root  Mean  Square  Error  (RMSE)  for  Model 
Output  Statistics  (MOS)  and  Persistence  for  Cloud 
Amount 


LFILING  HEIGHT 
O-MOS 

•  -PERSISTENCE 


O - - 

■IP! 


.1.  .  1. 

2  4 

FORECAST  LENGTH  (Hi 


Figure  27.  Hoot  Mean  Square  Terror  (RMSE)  for  Model 
Output  Statistics  (MOS)  and  Persistence  for  Ceiling 
Height 


here  with  the  forecaster-  consensus  results  in  Figure  18.  MOS  yielded  slightly 
lower  r-mse  results  over  the  2-vr  test  sample. 

MOS  probability  forecasts  of  cloud  cover  and  ceiling  height  compared  to  fore¬ 
caster  consensus  and  GEM  are  shown  in  Figures  28  and  29.  These  results  (ire 
comparable  to  the  rmse  results  that  suggest  that  1-  and  2-hr  usage  of  MOS  is 
of  questionable  value.  While  at  4-hr  and  beyond  it  will  generally  yield  better 
guidance  than  GEM  and  persistence  during  winter  storm  conditions.  In  fact,  MOS 
was  slightly  better  than  forecaster  consensus  for  both  variables  at  n-hr.  The 
percent  improvement  of  MOS  fi-hr  QPE  probability  forecasts  over  persistence  was 
42  percent  compared  to  a2  percent  for  the  forecasters  with  an  associated  rmse  of 
0.27  in.  which  compares  to  0.21  in.  for  the  forecasters. 
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Figure  28.  Percent  Improvement  in  Cumulative  p-score 
of  Model  Output  Statistics  (MOS)  Cloud  Amount  Forecasts 
vs  Persistence,  for  All  Forecasters  and  for  GFM 
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Figure  20.  Percent  Improvement  in  Cumulative  p-score 
of  Model  Output  Statistics  (MOS)  Ceiling  Height  Forecast 
vs  Persistence,  for  All  Forecasters  and  for  GFM 


3.3.4  IMPLICATION  OF  FORECASTER  EXPERIENCE 


One  outcome  of  these  tests  was  the  fairly  wide  range  in  individual  forecaster 
skill  that  resulted  in  all  aspects  of  the  experiment.  In  view  of  the  varying  levels 
of  forecasting  experience  among  the  participating  forecasters,  particularly  with 
regard  to  probability  forecasting,  an  index  of  forecasting  experience  was  estab¬ 
lished  in  order  to  evaluate  the  relationship  between  experience  and  skill.  The 
experience  index  (El)  was  defined  as  follows  for  the  numerical  forecasts: 

El  -  0.  6(OF)  +  0.  2(CFE)  +  0.  fi(MFE)  +  0.  4(MN)  +  0.  3(IF) 
and  for  probability  forecasts: 

El  =  0.  4(OF)  +  0.  2(CFE)  +  0.  8(MFK)  +  0.  (I(M N)  +  0.  4(IF)  , 

where 

OF  -  operational  forecasting  experience 

CFE  -  semester/year-long  college  forecast  exercises 

MFE  -  prior  Mesoscale  Forecast  Experiments 

MN  -  AFGL  Mesonetwork  Experiment  in  1970s 

IF  -  intangible  factor  based  on  perceived  day-to-day  interest  in  forecasting. 

Each  factor  was  quantified  in  terms  of  years  (or  equivalent  years)  of  experience. 
The  weights  were  assigned  to  reflect  relative  value  (as  judged  by  the  senior' 
author)  of  the  experience(s)  for  the  purposes  of  the  MFEs.  For  that  reason  the 
weights  assigned  to  prior  probability  forecast  experiments  (for  example,  MFE 
and  MN)  were  increased  and  OF  decreased  for  the  evaluation  of  probability  fore¬ 
casts. 

Figures  30  through  32  depict  the  simple  regression  lines  of  best  fit  between 
forecaster  error  (rmse)  and  the  El  for  the  numerical  forecasts  of  wind,  cloud 
amount,  and  ceiling  height  for  the  1 -,  2-,  4-.  and  n-hr  forecasts.  A"goodness- 
of-fit"  measure  (correlation  coefficient)  for  each  regression  line  is  also  shown. 

The  ceiling  height  results  are  for  the  1983  MFE  only.  With  the  c:  ception  of  the 
1-  and  2-hr  wind  forecasts,  the  anticipated  relationship  between  SKill  and  exper¬ 
ience  was  realized:  that  is,  decreasing  error  with  increasing  experience.  There 
is  also  reasonable  consistency  among  the  regression  lines  for  each  forecast  vari¬ 
able  in  that  the  slopes  closely  approximate  each  other-  (again  with  the  exception  of 
the  short-term  wind  forecasts)  and  the  y-intercepts  increase  with  increasing  fore- 
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Figur  e  30.  Hoot  Mean  Square  Krror  (RAISE)  for-  Mes- 
oseale  Forecast  Kxperiment  (MFK)  of  Forecasters  vs 
experience  [ndex  (FI)  for  Wind  Forecasts 


Figure  31.  Root  Mean  Square  Krror  (RAISE)  for 
Wesoscale  Forecast  Experiment  of  Forecasters 
vs  Experience  Index  for  Cloud  Amount  Forecasts 


Figures  33  and  34  show  the  regression  fit  of  the  skill  scores  vs  persistence 
and  El  for  cloud  amount  and  ceiling  height  probability  forecasts  for  the  full 
MFK  82-83  period.  Here  again,  the  strong  hint  of  experience  contributing  posi¬ 
tively  to  skill  is  evident,  in  this  case  increasing  skill  score  with  increasing 
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Figure  32.  Root  Mean  Square  Error  (RMSE)  for  Mesoscale  Forecast  Experiment 
(MFE)  of  Forecasters  vs  Experience  Index  (El)  for  Ceiling  Height  Forecasts 
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Figure  33.  Percent  Improvement  in  Cumulative  p-score  of  Mesoscale  Forecast 
Experiment  (MFE)  of  Forecasters  vs  Experience  Index  (El)  for  Cloud  Amount 
Probability  Forecasts 


Figure  34.  Percent  Improvement  in  Cumu¬ 
lative  p-score  of  Mesoscale  Forecast  exper¬ 
iment  (MFF)  of  Forecasters  vs  experience 
Index  (FI)  for  Ceiling  Height  Probability 
Forecasts 


experience.  Reasonable  consistency  is  maintained  among  the  regression  slopes, 
though  not  as  well  as  for  the  numerical  forecasts.  The  "  goodness-of-fit"  meas¬ 
ures  are  generally  higher  (across  the  board)  with  these  data  than  with  the  numer 
ical  forecast  results. 


4.  CONCLUSIONS 

An  MFF  was  conducted  over  a  2-yr  per  iod  using  the  AFGI.  Mel  DAS  facility 
and  data  from  20  winter  early  spring  east -coast  storm  episodes.  The  MFF  was 
structured  to  emulate  aspects  of  the  aviation  terminal  weather  support  functions 
in  base  weather  stations,  using  a  computer-driven  interactive  graphics  display 


system  of  the  type  envisioned  with  the  AWDS.  Data  available  to  the  MFK  fore¬ 
casters  included  that  which  will  be  part  of  the  initial  AWDS  and  that  which  could 
be  made  available  in  future,  expanded  AWDS  configurations.  The  Mi'll  was 
designed  to  assess  numerous  display  and  analysis  products  tailored  to  provide 
mesoscale  detail,  of  remotely-sensed  data  sources,  and  of  the  ability  of  fore¬ 
casters  to  prepare  certain  short-range  terminal  weather  forecasts. 

Each  participating  forecaster  completed  two  assessment  forms  after  each  of 
the  individual  forecast  episode  exercises.  The  first  form  addressed  the  value  of 
certain  display  products  and  data  sources  in  preparing  short-range  terminal  fore¬ 
casts  of  windspeed  and  direction,  total  cloud  amount,  ceiling  height,  and  (1-hr 
QPF.  The  second  form  dealt  with  the  relative  difficulty  in  forecasting  each  of  the 
four  elements  accompanied  by  a  discussion  of  the  products 'data  sources  important 
to  the  forecast  preparation  process.  Over  the  course  of  the  20-episode  test,  there 
were  110  sets  of  assessment  forms  completed.  In  addition,  a  statistical  verifica¬ 
tion  of  forecaster  performance  was  conducted  for  the  numerical  and  probability 
forecasts  that  were  generated. 

The  products  judged  to  be  most  useful  in  the  preparation  of  short-range 
terminal  forecasts  included:  (1)  conventional  geographic  data  displays  presented 
simultaneously  as  four  quadrant  panels  on  one  screen,  (2)  regional  scale  surface 
analyses,  plots,  and  data  listings  of  basic  variables,  (.3)  satellite-based  trajec¬ 
tory  technique,  (4)  tailored  plot  displays  such  as  the  stat ion -model  times-series 
displays,  and  (5)  mapped  displays  of  forecast  guidance  derived  from  the  \’IUC 
LFM  model.  Products  such  as  log  p-skew  T  soundings,  analyses  of  derived  sur¬ 
face  and  upper-air  variables  such  as  vorticity  or  temperature  advection,  3-D 
trajectory  guidance  and  GEM  were  found  to  be  of  very  limited  value  due  to  their 
lack  of  timeliness  and/or  poor  horizontal  representativeness. 

The  current  operational  availability  of  surface  and  sounding  data  (in  3-D 
space  and  time)  is  not  compatible  with  the  needs  for  mesoscale  data  for  short- 
range  terminal  forecasting  purposes.  This  incompatibility  forces  the  forecaster 
to  rely  most  heavily  on  data  presented  in  fairly  basic  form  (listings,  plots,  and 
simple  analyses  of  basic  variables),  which  can  be  interpreted  and  filtered  sub¬ 
jectively  in  order  to  build  an  unde rstanding  of  the  atmospheric  processes  taking 
place  in  the  forecast  region.  The  tailored  products  that  were  most  relied  on 
retained  or  highlighted  mesoscale  aspects  of  the  basic  variables  associated  with 
the  extratropical  storm  systems  being  dealt  with  in  the  MFE. 

Half-hourly  imagery  (visible  and  IE)  from  geosynchronous  weather  satellites 
like  GOES  represent  the  single -most  important  data  source  for  short-range 
aviation  terminal  forecasting.  The  MFE  forecasters  relied  more  heavily  on  it  to 
prepare  their  cloud  cover,  fi-hr  QPF,  and  ceiling  height  forecasts  than  any  other 
available  source,  The  ability  to  manipulate  GOES  imagery  interactively  on  the 


graphics  display  provides  a  wealth  of  qualitative  and  quantitative  guidance  for 
forecasting  purposes.  This  capability  includes  time-series  looping,  rapid  chan¬ 
nel  switching  (visible  to  IK  and  vice  versa),  selective  color  enhancement  and 
remapping,  and  overlaying  conventional  data  plots  and  analyses  on  the  imagery's 
geographic  coordinates.  Manually  digitized  radar  (MDR)  national  summaries, 
objectively  analyzed  and  displayed  on  the  color  monitor,  were  found  to  be  most 
valuable  for  fi-hr  QPF,  and  to  a  lesser  extent,  ceiling  height  forecasting  purposes 
The  most  serious  limitation  or  drawback  of  MDR  data  was  its  sporadic  availabil¬ 
ity  for  various  and  sundry  reasons.  This  created  an  erosion  of  confidence  in  it 
as  a  reliable  data  source. 

Consistent  with  the  results  reported  in  Chisholm  et.  al,  *  ceiling  height  was 
found  to  be  the  most  difficult  element  to  forecast  due  to  its  inherent  variability 
during  storm  situations,  its  sensitivity  to  terrain  considerations,  and  the  limits 
of  our  ability  to  fully  observe  and  report  it  from  either  ground-based  or  satellite 
perspecti ves.  Terminal  wind  conditions  were  deemed  to  be  the  second  most 
difficult  variable  to  forecast.  The  forecasting  of  (i-hr  QPF  is  aided  more  than 
the  other  elements  by  the  availability  of  half-hourlv  GOFS  imagery  and  MDI! 
displays.  As  a  result  it  was  viewed  to  be  the  second  easiest  variable  to  forc  e  .s', 
while  total  cloud  amount  was  judged  easiest,  in  most  cases,  forecasters  use  1 
GOES  imagery  almost  exclusively  to  forecast  total  cloud  amount,  especially  for 
periods  less  than  4  hr. 

The  2-yr  forecast  experiment  resulted  in  over  1300  forecasts  of  each  vari¬ 
able  for  each  forecast  interval.  A  statistical  ver  ification  of  the  forecasters' 
numerical  and  probability  forecasts  was  conducted  with  comparisons  made  to 
persistence,  sample  climatology,  and  two  forecast  guidance  techniques;  namely, 
MOS  and  GEM.  While  the  primary  evaluation  involved  grouped  forecaster  per¬ 
formance  (consensus),  individual  performance  and  its  implications  were  also 
evaluated.  The  MFF  forecasts  yielded  superior  rmsc  compared  to  persistence 
for  each  variable  (winds peed  and  direction,  total  cloud  amount,  ceiling  height, 
and  fi-hr  QPF)  for  all  forecast  intervals  (1,  2,  4,  and  n  hr).  At  4  hr,  the  improve¬ 
ment  over-  persistence  ranged  from  about  20  percent  for  wind  and  ceiling  height  to 
about  30  pf'rcrnt  for  total  cloud  amount.  The  forecasters  probability  forecasts 
were  substantially  better  than  persistence  probability  at  all  forecast  intervals 
and  showed  skill  vs  sample  climatology,  except  at  the  n-hr  interval.  While  GFM 
yielded  skill  relative  to  persistence  probability,  in  general  it  yielded  higher  rmsc 
scores  than  did  persistence  in  its  numerical  form.  Its  guidance  value  was  deemed 
to  be  minimal  in  "heavy"  weather  episodes  of  the  type  user!  in  the  MFF.  Lastly, 
MOS  was  found  to  provide  useful  guidance  for  forecast  intervals  of  4  hr  and  be¬ 
yond,  particularly  as  regards  wind  forecasts.  The  variation  in  individual  fore¬ 
caster  skill  was  found  to  he  fairly  substantial  and  at >  nbut able,  in  part,  to  fore¬ 
casting  experience. 
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Based  on  the  2-vr  MEL,  the  following  conclusions  are  reached  vis-a-vis  the 
use  of  interactive  graphics  display  systems  for  short-range  terminal  forecast 
preparation  and  monitoring: 

(1)  Software  for  user-specified  data  plotting  and  analysis  in  tailored  form 
(for  example,  four-panel  plot  and  analysis  displays,  stat ion -model  time-series 
displays)  is  essential  to  allow  selective  hut  substantial  amounts  of  basic  data  to 
be  viewed  and  manipulated  interactively. 

(2)  Fundamental  to  (1)  is  the  presumed  availability  of  a  recent  history  file 
(at  least  the  latest  24  hr)  of  basic  surface  and  upper-air  observations  in  the 
resident  computer  terminal,  such  that  products  can  be  prepared  and  displayed 
within  a  minute  or  so  of  its  request. 

(3)  The  current  observational  density  (in  space  and  time)  for  both  surface 
and  upper-air  operational  data  sources  seriously  limits  the  extent  to  which  meso- 
scale  features  can  he  detected  and  used  for  short-range  forecas  ng  purposes.  As 
a  result  little  value  can  be  found  in  computing  and  displaying  more  complex  de¬ 
rived  fields  such  as  moisture  convergence ,/advection,  vorticitv,  and  divergence. 

(4)  A  key  to  progress  in  improved  short-range  terminal  forecast  support  lies 
in  geosynchronous  weather  satellites,  which  not  only  can  provide  continuously 
repeating  views  of  a  geographical  area  to  provide  valuable  imagery  information 
for  t  ranslution /extrapolation  of  clouds  and  precipitation  features,  but  also  fills 

in  the  data  voids  between  widely  separated  surface  and  radiosonde  stations.  With 
the  continued  development  of  GOES  temperature  and  water  vapor  sounder  capabil¬ 
ities,  the  importance  of  satellites  for  short-range  forecasting  support  globally 
will  increase  over  the  next  several  years. 

(5)  Although  there  is  a  wide  range  of  research  underway  for  modeling  small- 
scale  weather  systems  (both  physically  and  numerically),  we  are  still  many  years 
away  from  the  widespread  practical  application  of  such  models  in  operational 
forecasting  in  view  of  the  required  (and  generally  unavailable)  data  and  computer 
capacity.  Short-range  forecasting  must  continue  to  focus  on  detecting,  tracking, 
and  extrapolating  the  movement  and  evolution  of  mesoscale  systems.  Resident 
software  in  an  interactive  graphics  system  should,  therefore,  be  tailored  to 
presenting  weather  depictions  as  specified  by  the  forecaster-user.  They  should 
be  capable  of  incorporating  all  available  operational  data  sources,  especially  those 
like  M DR,  which  often  provide  vital  information  between  conventional  surface 
observation  sites, 

(fi)  Training  for,  and  familiarization  with,  the  use  of  an  interactive  system 
is  essential.  Like  any  advance  in  technology,  there  will  be  a  "learning"  period 
during  wdtich  forecaster  performance  (including  forecast  skill)  will  be  somewhat 
degraded.  The  speed  with  which  familiarity  with  the  system's  capabilities  and 
requirements  will  be  acquire  1,  will  vary  due  to  any  number  of  human  behavior 


factors.  Given  the  general  educational  background  and  basic  weather  forecasting 
training  provided  to  all  AWS  forecasters,  the  transition  period  from  the  current 
manual  mode  to  a  more  automated  mode  using  interactive  graphics  systems 
should  be  short  and  the  new  skills  needed  easily  acquired. 

(7)  The  trend  towards  "user-friendly"  interactive  procedures  (touch  screen, 
etc.),  while  advantageous,  should  not  be  overdone.  Although  the  keyboard  entry 
methodology  of  MelDAS  is  rudimentary,  and  at  first  glance  may  appeal-  to  be  a 
little  complicated,  familiarity  with  its  structure  and  options  is  acquired  with 
reasonable  speed  even  by  those  of  us  who  matured  in  the  pre-eleetronic  wizardry 
days.  It,  and  many  other  systems,  has  extensive  built-in  default  parameters 
(for  example,  contour  intervals  for  each  parameter),  which  simplify  keyboard 
entry.  Again,  the  key  to  effective  use  of  the  system  does  not  lie  in  its  keyboard 
entry  method  but  rather  on  the  resident  data  and  software  in  it  that  can  generate 
the  display  products  required  by  the  forecaster  for  base  weather  station  support 
requirements. 
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Appendix  A 

1983  Episodes 

Forecast  Experiment  11:  At  15  GMT,  25  October  1982,  a  well  developed, 
occluded  cyclone  was  centered  over  Cape  Hatteras  beneath  a  500  mb  low  (Fig¬ 
ure  Al).  Light  to  moderate  rain  was  falling  from  southeastern  New  York  south¬ 
ward  to  the  Carolinas.  With  little  further  development  expected  as  the  cyclone 
moved  slowly  toward  the  northeast,  the  main  forecast  problem  consisted  of  timing 
the  arrival  of  the  precipitation,  (i-hr-  precipitation  amount  and  ceiling  height,  fore¬ 
casts. 


Figure  Al.  General  Weather  Situation  for  MFF  Case  No.  11:  (a)  Surface  Pres¬ 
sure  Analysis  for  1500  GMT,  25  Ortober  1982  and  (b)  500-mh  Height  Analysis  for 
1200  GMT.  25  October  1982 


Forecast  Experiment  12:  At  15  GMT,  5  November  1982,  a  strong,  but  slow, 
eastward-moving  cold  front  was  oriented  north  to  south  across  eastern 
New  England  (Figure  A2).  Rainfall  was  occurring  ahead  of  the  front  with  scat¬ 
tered  showers  behind  the  front.  More  than  an  inch  of  rain  fell  as  the  front 
marched  eastward  in  a  band  from  Quebec  southward  through  Vermont  and  eastern 
New  York  and  eastern  Pennsylvania.  With  the  approaching  front  within  50  to  f!0  km 
of  Boston  and  Providence  the  timing  of  FROPA  and  its  associated  wind  shift  and 
improving  ceiling  and  cloud  conditions  were  of  most  concern. 
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Figure  A2.  General  Weather  Situation 
for  MFi:  Case  No.  12:  (a)  Surface  Pres¬ 
sure  Analysis  for  1500  GMT,  5  November 
1982  and  (b)  500-mh  Height  Analysis  for 
1200  GMT.  5  November  1982 


Forecast  l-'xperiment  13:  The  I!  G11T,  13  \ovemhcr  lf'82,  su:  1 
analyses  /Figure  A3)  showed  a  surface  pressure ■  trough  associated 
slowly  moving  cold  front  approaching  western  New  Knglund.  With  ror 
advert  ion  along  the  cold  front,  plentiful  moisture,  and  a  we  ik  frontal  w 
inc  dong  the  Ne.v  Jersey  roast  the  most  immediate  forecast  problen  w 
precipt'ation  amoun’  and  the  strong  winds  that  were  a  Ire  d\  oceurrtng  t 
Boston  reported  soutlierdy  wind  at  2*5  knots,  gusts  to  .38  knots).  In  the  1 
periods  timing  ‘hr  frontal  passage  and  the  associated  wind  shift,  ,n.i  -a 
i iam’  rainfall  is  do  nrring  heiting  the  front,  t;-hr  nt  i  eipitaton  miount 
greatest  forecast  I'OMia  rns. 
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Figure  A3.  General  Weather  Situation 
for  MFK  Case  No.  13:  (a)  Surface  Pres¬ 
sure  Analysis  for  0(100  GMT,  13  Novcwn - 
her  lf'82  and  {!>)  500 -mb  Ilelgh1  Anilvsis 
for  0000  GMT,  13  Novembe .  Ihlk: 


Forecast  Experiment  14:  The  surface  pressure  analysis  at  12  GI\IT, 

15  January  1983,  (Figure  A4)  showed  a  broad  area  of  low  pressure  moving  across 
rhe  mid-Atlantic  states.  A  primary  low  was  located  in  northwestern  Pennsylvania 
with  a  secondary  developing  near  the  Maryland  coast.  With  a  potent  500-mb 
trough  (Figure  A4)  and  strong  PVA  overspreading  the  mid-Atlantic  coast  the 
secondary  would  likely  become  a  major  snowstorm  for  the  A'ortheast.  With  the 
potential  for  a  major  snowstorm  approaching  the  Northeast,  heavy  precipitation 
amounts,  strong  winds,  and  low  ceilings  were  likely  forecast  problems. 
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Figure  A4.  General  Weather  Situation 
for  MFK  Case  No.  14:  (a)  Surface  Pres¬ 
sure  Analysis  for  1200  GMT,  15  January 
1983  and  (b)  500-mb  Height  Analysis  for 
1200  GMT,  15  January  1983 
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Forecast  Experiment  1G:  At  lfi  GMT,  11  Feb  1983,  an  intensifying  cyclone 
was  located  along  the  Carolina  coast  (Figure  AG)  while  a  very  strong  cold  high 
pressure  system  (1040  mb)  was  centered  in  eastern  Canada  extending  southeast¬ 
ward  over  New  England.  The  strong  pressure  gradient  between  these  two  systems 
generated  a  strong,  moist  easterly  flow.  The  combination  of  the  moist  easterly- 
flow  and  strong  warm  advection  at  850  and  700  mb  were  combining  to  produce 
extreme  snowfall  rates  (1  to  5  in.  /hr).  Satellite  imagery  showed  the  cloud  shield 
steadily  moving  toward  the  NNE  suggesting  the  potential  for  heavy  snow  in 
southern  New  England.  With  near  blizzard  conditions  approaching  from  the  south, 
winds,  heavy  snow  and  very  low  ceilings  were  likely  forecast  problems. 


Figure  AG.  General  Weather  Situation 
for  RIFE  Case  No.  1G:  (a)  Surface  Pres¬ 
sure  Analysis  for  1G00  GMT,  11  February 
1983  anti  (b)  500-mb  Height  Analysis  for 
1200  GMT,  11  February  1983 
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Forecast  Experiment  17:  The  23  GMT,  14  Jan  1983  surface  pressure  analysis 
(Figure  A7)  showed  an  occluded  low  centered  over  northern  Ohio,  Clear  skies 
still  covered  much  of  New  England  at  23  GMT  while  significant  weather  was  still 
several  hundred  miles  to  the  west.  Thus  the  main  forecast  problem  was  increas¬ 
ing  cloud  amount  and  gradually  lowering  ceilings  in  the  later  forecast  periods. 
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Figure  A7.  General  Weather  Situation 
for  MFK  Case  No.  17:  (a)  Surface  Pres¬ 
sure  Analysis  for  2300  GMT,  la  January 
1983  and  (h)  500 -mb  Height  Analysis  for 
0000  GM  T.  10  January  1983 


Forecast  Experiment  18:  A  cold  front  in  eastern  New  York,  associated  with 
a  low  pressure  system  over  southeastern  Canada  at  1(>  GMT,  23  Feb  1983,  was 
moving  steadily  eastward  toward  southern  New  England  (Figure  A8).  The  com¬ 
bination  of  the  approaching  cold  front  and  a  low  pressure  system  developing  over- 
eastern  North  Carolina  whose  precipitation  shield  was  expected  to  graze  southern 
New  England,  bringing  a  period  of  light  rain  to  Boston  and  Providence.  The  main 
foreras  problem  was  timing  the  cold  frontal  passage  and  its  wind  shift  and  the 
worsening  ceiling  conditions  associated  with  the  light  rain. 


Figure  A8.  General  Weather  Situation 
for  AIl-'F  Case  No.  18:  (a)  Surface  Pres¬ 
sure  Analysis  for  1 900  GMT,  23  February 
1983  and  (b)  aOO-rnb  Height  Analysis  for 
1200  GMT,  23  February  1983 
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Forecast  Kxperimont  IP:  At  14  GMT,  2  Mar  1983,  an  intense  cyclone  was 
located  approximately  300  km  off  the  Delaware  coast  moving  toward  the  northeast 
(Figure  AO).  A  large  rain  shield  that  had  already  dumped  over  an  inch  of  rain  at 
Itoston  and  Providence  stretched  from  Maine  southward  to  Virginia.  As  this  large 
ocean  storm  continued  to  move  slowly  toward  the  northeast  with  a  continuation  of 
moderate  rain,  (i-hr  precipitation  amount  was  the  main  forecast  problem. 


Figure  A9.  General  Weather  Situation 
for  MFF  Case  No.  19:  (a)  Surface  Pres¬ 
sure  Analysis  for  1400  GMT,  2  March 
1983  and  (b)  500-mb  Height  Analysis  for 
1200  GMT,  2  March  1983 
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Forecast  Experiment  20:  The  15  GMT,  21  Mar  1983,  surface  pressure 
analysis  (Figure  A10)  showed  an  area  of  low  pressure  oriented  from  southwestern 
Pennsylvania  southeastward  to  northern  Virginia.  A  weakening  occluded  cyclone 
was  located  in  northwestern  Pennsylvania  with  a  developing  frontal  wave  at  the 
triple  point  in  northern  Virginia.  In  the  early  periods  of  the  experiment  the  main 
forecast  problem  was  timing  the  approach  of  overcast  conditions  and  lowering 
ceilings.  Later,  it  was  timing  the  approaching  warm  front  and  its  heavy  rain 
showers  and  eventually  a  cold  frontal  passage  and  the  associated  wind  shift. 
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Figure  A10.  General  Weather  Situation 
for  Mi'll  Case  \'o.  20:  (a)  Surface  Pres¬ 
sure  Analysis  for  1500  GMT.  21  March 
1983  and  (b)  500-mb  Height  Analysis  for 
1200  GMT,  21  March  1983 


-.'S'.-.'' 


Product  Usefulness  Assessment 


Case  No. _  Case  Date _  Forecaster _ 

Forecast  Stations  BOS  and  _ 

First  Forecast  Time _ Z  Last  Forecast  Time _ Z 

Circle  the  products  you  used  in  this  case: 

MS  GT  MR  PF  KZ  TS  FA  FK-THA  FK-MOS 
FK-GUI  BS  1A 

1.  Most  Useful 

A.  What  product  was  most  useful  to  you0 

B.  Why  was  it  so  useful? 

C.  How  did  you  use  it0 

D.  In  what  situations  was  it  most  useful0 
F.  In  what  situations  was  it  not  useful0 

2.  Least  Useful 

A.  What  product  was  least  useful0 

B.  What  made  it  so  useless0 

C.  In  what  situations  was  it  most  useful0 

D.  In  what  situations  was  it  least  useful0 

E.  How  could  the  product  be  improved0 


Mesoscale  Forecast  Variable  Assessment 


Case  No. _  Case  Date _  Forecaster _ 

Forecast  Stations  BOS  and  _ 

First  Forecast  Time _ Z  Last  Forecast  Time _ Z 

Ranking  of  Forecast  Variable,  in  order  of  difficulty  using  facilities  available 
through  McIDAS  and  MFF 

A.  The  most  difficult  variable  to  forecast  was  _ 

beyond _ hours,  I  had  little  confidence  in  my  forecasts. 

Was  satellite  imagery  useful'’  If  so,  how? 


Was  MDR  useful  0 


If  so,  how  ? 


Were  the  mesoscale  plot  and  analysis  routines  useful? 


If  so, 


how 


Was  guidance  information  useful? 

If  so,  which  guidance  (GFM,  MOS,  LFM  Guidance,  3-D  Trajectory)  and  how 


Were  the  Wisconsin  trajectory  models  useful?  If  so,  how  ? 


Was  this  forecast  variable  affected  by  local  (non-translatory)  factors? 
If  so,  how  did  you  factor  that  into  your  forecast0 


B.  The  next  most  difficult  variable  to  forecast  was _ 

beyond _ hours,  I  had  little  confidence  in  my  forecasts. 

Was  satellite  imagery  useful?  If  so,  how? 

Was  MDR  useful0  If  so,  how0 

Were  the  mesoscale  plot  and  analysis  routines  useful?  If  so,  how? 


W'as  guidance  information  useful0 

If  so,  which  guidance  (GFM,  MOS,  LFM  Guidance,  3-D  Trajectory)  and  how 


Were  the  Wisconsin  trajectory  models  useful  ?  If  so,  how  9 

Was  this  forecast  variable  affected  by  local  (non-translatory)  factors0 
If  so,  how  did  you  factor  that  into  your  forecast? 

C.  The  next  most  difficult  variable  to  forecast  was _ _ ; 

beyond _  hours,  I  had  little  confidence  in  my  forecasts. 

Was  satellite  imagery  useful0  If  so,  how0 

Was  MDR  useful0  If  so,  how0 

Were  the  mesoscale  plot  and  analysis  routines  useful0  If  so,  how0 

Was  guidance  information  useful? 

If  so,  which  guidance  (GEM,  MOS,  LFM  Guidance,  3-D  Trajectory)  and  how? 

Were  the  Wisconsin  trajectory  models  useful?  If  so,  how? 

Was  this  forecast  variable  affected  by  local  (non-translatory)  factors? 

If  so,  how  did  you  factor  that  into  your  forecast0 

D.  The  easiest  variable  to  forecast  was _ ; 

beyond _ hours,  I  had  little  confidence  in  my  forecasts. 

Was  satellite  imagery  useful0  If  so,  how0 

Was  MDR  useful0  If  so,  how  ? 

Were  the  mesoscale  plot  and  analysis  routine  useful0  If  so,  how? 

Was  guidance  information  useful0 

If  so,  which  guidance  (GEM,  MOS,  LFM  Guidance,  3-D  Trajectory)  and  how0 

Were  the  Wisconsin  trajectory  models  useful?  If  so,  how0 

Was  this  forecast  variable  affected  by  local  (non-translatory)  factors0 
If  so,  how  did  you  factor  that  into  your  forecast? 
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